- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 1
Read review
Paper summary
This paper studies user-memory management in long-term conversations and proposes the SaliMory framework. It organizes memory into factual memory, preference memory, and working memory, and uses the Saliency Filter, MemBooster, and MemUtilizer for memory filtering, updating, and query-time utilization. The paper combines stage-wise rewards, GRPO, and role-specific pairwise training, and reports promising results on LoCoMo, LoCoMo-P13n, LongMemEval, and internal data. However, the current results are not yet sufficient to support the paper’s central claims regarding role-level credit assignment and the effective isolation of optimization signals across different stages. The main concerns involve gradient attribution under a shared policy, the state consistency of contrastive examples, reward coverage, data splitting and evaluation reliability, and the fairness of the baselines and statistical reporting.
Critical strengths
(1) The problem is clearly important in practice and covers three key aspects of long-term memory: memory formation, maintenance, and utilization. The paper evaluates not only QA performance, but also personalization quality, memory factuality, vagueness, and utilization, which is more comprehensive than reporting final answer accuracy alone.
(2) Extensive experiments across multiple datasets, reward and contrastive-training ablations, efficiency analysis, and partial validation using an independent evaluator.
Critical weaknesses
(1) Role-level credit assignment is not sufficiently established by the current optimization objective.
The three roles use a shared policy (), while GRPO computes the group-relative advantage for each complete rollout based on the total reward: Rtotal = R1 + R2 + R3. Therefore, the current description does not explain why the main GRPO gradient would not simultaneously affect the Filter, Booster, and Utilizer. Nor does it demonstrate that the role-specific contrastive losses are sufficient to eliminate cross-role interference caused by shared parameters. In addition, the final () measures the quality of the complete memory bank, which is not equivalent to the local contribution of each intermediate memory-writing action.
(2) The contrastive pairs may not be constructed under matched conditions, weakening the causal-isolation argument.
Different Booster rollouts may produce different memory states at early steps, while the paper constructs winner/loser pairs based on the final R2. A better final result does not necessarily imply that every intermediate memory write was better. For the Utilizer, a Jaccard threshold of τsim=0.15 only guarantees a certain degree of set overlap. It does not guarantee that the key facts, item ordering, prompt length, or context conditions are identical. Consequently, the difference in utilization quality may still be caused by upstream memory differences rather than by the Utilizer itself.
In addition, the current objective is more accurately described as a weighted DPO-style pairwise logistic loss; the paper does not show the reference-policy log-ratio term used in the standard DPO formulation.
(3) The data split and personalization-label definitions do not sufficiently rule out leakage and evaluation bias.
The paper reports 1,108 additional questions and a 60/20/20 split, but does not clearly explain whether the split is performed by user, conversation, session, or QA pair. If the same user or conversation history appears across different splits, the reported test results may overestimate generalization to unseen users.
The generation process for the additional questions, the proportion subjected to human review, and the quality-control procedure are also insufficiently transparent. Furthermore, whether Underuse and NA are determined using a shared original-history reference or each method’s own memory bank directly affects the fairness of the comparison.
(4) LLM-based evaluation, statistical robustness, and baseline fairness remain insufficient.
GPT-4o is used as the primary judge, while another judge is used only for cross-validation on a limited subset. Category-level confusion patterns, the number of human-evaluated examples, the sampling procedure, and confidence intervals are not sufficiently reported. The paper also mainly reports point estimates and lacks results across multiple random seeds, user- or conversation-level bootstrap estimates, and paired significance tests.
In addition, the policy model, frozen generator, judge, prompts, token budget, memory budget, and serving environment may not be consistent across methods. Therefore, the reported gains may not be attributable solely to the memory architecture.
Questions for authors
(1) The authors should clearly specify parameter sharing, token-level loss masks, gradient routing, and the update scope of each objective. They should also conduct ablations comparing global GRPO, role-specific rewards without contrastive training, role-specific contrastive training, and the full method, in order to identify the source of the reported gains.
(2) The authors should report the construction procedure and acceptance rate of the preference pairs. They should also add controlled experiments with fixed preceding states, shared prefixes, fixed memory contents, and different similarity thresholds, together with an ablation without the similarity gate.
(3) The authors should clearly specify the split granularity, user-level isolation protocol, memory-reset rules, and human-review procedure. They should also use a method-independent personalization-opportunity reference.
(4) The authors should report baselines using matched generators and matched computational budgets, provide the complete training and inference costs, and clearly distinguish official implementations from author reimplementations.
(5) QA is not the highest for SaliMory on every LongMemEval subset; the paper should avoid claiming universal superiority. In the R1-only ablation, Vague is 28.3 while Bad Personalization is 4.5; these values should not be conflated. Public-benchmark and internal-traffic Vague/Factuality values differ substantially and should be discussed separately. The semantics of Apply, memory insertion/deletion/merging, conflict handling, capacity limits, pair filtering, and the relationship between β and temperature η need to be specified for reproducibility.
5 comments
Rebuttal
W3 / Q3. Split, LoCoMo-P13n construction, Underuse/NA reference.
Split granularity and LongMemEval's role. The split is over episodes, so questions about the same LoCoMo history can appear in both train and test. This does not bias our results. (i) Questions over the same history span different types (single-hop, multi-hop, temporal, open-domain, recommendation, implicit personalization) that draw on different evidence and require different reasoning, and memory is rebuilt from scratch in every episode, so no answer or memory carries over from training. (ii) LongMemEval-s gives every question its own user history, so no test history is seen in training; it is trained and evaluated separately, with no cross-training, and SaliMory's gains hold there (69.1 QA / 20.8 Good, Table 8 in the paper). (iii) On a new split of LoCoMo by conversation, where test users and histories are entirely unseen, SaliMory's gain over the untrained policy holds (+10.8 QA / +18.2 Good). Hyperparameters and checkpoints are selected on the validation partition only.
The 1,108 Cat 6–7 pairs were generated by the 4-step semi-automatic pipeline of Appendix A.1 in the paper via Llama3.3-70b-instruct. 100% were manually reviewed by a linguistic expert, who rejected or corrected pairs not supported by the conversation. Good Personalization is computed over a fixed question set, so its denominator does not depend on what is stored and a method cannot raise it by storing less: missing memories turn into NA or Underuse and lower Good. Memory is reset between episodes for every method. We will add these details to Appendix A.1.
W4 / Q4. Statistics, judges, matched budgets, cost.
The gains come from the training algorithm, not only the architecture: with the same architecture, prompts, generator, and budget, the full method improves over GRPO-only by +4.3 QA / +13.0 Good (Table 2 in the paper), and over our new untrained-policy baseline (same architecture, untrained Qwen3.5-9B) by +12.5 QA / +21.2 Good (72.9 vs. 60.4; 39.8 vs. 18.6).
| Comparison (LoCoMo) | Δ QA | 95% CI | Δ Good | 95% CI |
|---|---|---|---|---|
| SaliMory vs. Mem-R1 | +10.2 |
| +23.5 |
| | SaliMory vs. GRPO only | +4.3 |
| +13.0 |
| | SaliMory vs. Zero-shot Agentic | +2.0 |
| +33.2 |
|
New analysis: bootstrap 95% CIs over test episodes. The Δ values are computed from Tables 1 and 2 in the paper.
Judges. Appendix A.3 in the paper reports human agreement for the final-response judge, and Table 5 in the paper shows identical rankings under a second judge, Claude 3.5 Sonnet (ρ = 1.00). We have now also validated the intermediate judges (new analysis): across judge families (GPT-4o vs. Gemini 3.1 Pro), response-level signals agree with κ = 0.62–0.72 and utilization with r = 0.70; against human ratings, utilization reaches r = 0.78 and the memory-quality signals r = 0.61–0.66 . Budgets. All baselines in Table 1 of the paper are run with the same frozen generator, judge, and prompts. The one exception is Zero-shot Agentic, which uses Gemini-3-Flash for memory operations; we will flag this in the table caption.
| Method | Training compute | Inference tokens / episode | Latency / episode |
|---|---|---|---|
| Mem-R1 | Not reported (4× H100, two RL agents), est 3.4 hrs per epoch | 12~18k | not reported per episode. Per-call p50 only (Appendix G in the paper): manager ≈2.0s, answer 0.34s |
| Zero-shot Agentic | — | 12~18k | 38.7s (38s formation + 0.69s/query,including network latency) |
| SaliMory | 8× A100, 1.14 s per episode (Table 3 in the paper) | 8~10k | 0.76s (0.5s formation + 0.26s/query) |
New: inference tokens measured in our setup. SaliMory training cost and all latencies: Table 3 in the paper. Mem-R1 entries: from the Mem-R1 paper and our estimate.
Q5. Claim scoping and specification.
We will make all of these changes: (i) scope the QA claim to “best QA among trained memory methods, on par with a frontier zero-shot agent” and fix the bold formatting in Table 8 in the paper; (ii) separate the R1-only ablation's Vague rate (28.3) from Bad Personalization (4.5); (iii) discuss public-benchmark and internal-traffic Vague/Factuality separately; (iv) specify Apply semantics (insertion, update, conflict handling, capacity limits), pair filtering, and a single symbol for β/η in §3.2–3.4 and Appendix A.6.
We sincerely thank you for a precise set of questions; addressing them has tightened the paper. Looking forward to your responses!
Thanks for the detailed response.
The explanation on shared GRPO update, new ablations, and the conversation-disjoint LoCoMo experiments are helpful to address my concerns.
However, my remaining concern lies in whether the preference pairs can isolate each role’s contribution. For the Booster, a higher final memory-bank score may reflect better writes at other turns, so it is unclear whether the preferred write at the paired turn is actually better or not. For the Utilizer, a higher score may come from the access to more relevant facts or from the use of same facts. In both cases, the comparison leaves open issue whether the preferred output earned its higher score through the role’s own decision.
Additionally, when deciding whether there was an opportunity to personalize the answer, does the judge use the original conversation history, or the memories each method actually stored?
Rebuttal
Thank you for the encouraging assessment. We address each point below.
In Responding to Weakness and Questions
W1 / Q1. Role-level credit assignment under a shared policy.
In SaliMory, memory is not optimized in isolation. Following the cognitive view that memory has value only through use, the design enables a loop: good memory leads to better utilization and better downstream responses, and downstream performance in turn guides what to store and how to use it. The roles are intentionally coupled through the downstream task, and separately supervised where attribution is needed. Memory formation, maintenance, and utilization matter only through the final response, so the shared GRPO term ties all roles to that outcome. Each role additionally receives a signal that measures its own output.
Role-specific-advantages row: new experiment. Other rows: from Table 2 in the paper.
On R2 being bank-level: booster pairs are formed per turn. For each utterance, the outputs of the R2-best and R2-worst rollouts are paired, and the loss is weighted by their R2 gap. Turns where both rollouts wrote the same output contribute nothing, so the update concentrates on the write decisions that differ. We will add this specification to §3.4.
W2 / Q2. Matched conditions; τ_sim; DPO terminology.
Why decompose the winner selection (illustrative episode, G = 4).
Illustrative numbers to explain the selection rule, not measured results.
Selecting by total reward would pair g3 against g2 and teach the booster g3's vague memories, even though g3 won only because the frozen generator answered well. Instead, SaliMory selects by each role's own score: the booster pairs g0 vs. g2 (gap 0.23), and the utilizer pairs g1 vs. g2 only if their banks overlap (Jaccard > τ_sim).
Matched conditions. Booster pairs share the episode, the conversation, and the utterance, and the loser's output is re-scored under the winner's prompt, so both writes are compared on the same input. Utilizer pairs share the question and conversation, and are kept only when the two memory banks overlap. Low-overlap pairs are dropped, and the utilizer still receives its continuous utilization reward through GRPO.
New: pair statistics logged from our training runs.
New τ_sim ablation. The 0.15 row is the submitted setting, from Table 1 in the paper.
Terminology. We agree: the objective is a gap-weighted pairwise logistic loss (DPO-style, without the reference log-ratio). The reference policy enters through the GRPO KL term (coefficient 0.08). We will rename it. η in Table 7 of the paper is the same parameter as β in Eqs. 4 and 6 (β = η = 0.15, scaling the pairwise logit); we will use a single symbol. Rollouts are sampled at temperature 0.5, which we will add to Table 7. For reproducibility, we also released the code: https://anonymous.4open.science/r/SaliMory-4D59.