- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper presents UpliftMem, a framework for learning agent memory retrieval from execution feedback. A trainable scorer predicts the utility gain of a complete memory set relative to execution without memory, while the task executor remains frozen. An expected value of sample information (EVSI) criterion selects additional training executions under a limited rollout budget, and their outcomes supervise the scorer. The system also validates newly written memories. Experiments on ALFWorld, WebShop, BigCodeBench, and MemSyco-Bench include fixed-store comparisons, training ablations, and matched probe-budget evaluations.
Critical strengths
The paper addresses a retrieval problem in agent memory and provides useful evidence. Selecting memories according to their execution benefit aligns retrieval with the agent’s actual objective. The fixed-store experiments help isolate retrieval improvements, while the acquisition comparisons use the same warm-up checkpoint and match probe budgets. These controls support the practical value of the proposed approach in the evaluated settings and help distinguish retrieval and feedback-selection improvements from changes in memory content or additional executions alone.
Critical weaknesses
W1. Evidence with stronger agents is limited.
The evaluation covers only Qwen3-4B and Qwen3-8B, leaving unclear whether the benefits persist with stronger or different-family executors. This matters because the value of external guidance may change with the executor’s existing capabilities. For example, the ALFWorld unseen improvement over the no-memory baseline is 20.91 percentage points for 4B but 2.23 points for 8B (Table 1). This does not establish a general decline with model strength, but motivates checking the method’s incremental value with a stronger executor and competitive memory baselines.
W2. The portability and adaptation cost of the learned scorer are unclear.
The scorer is adapted separately for each executor, so the experiments demonstrate the applicability of the training procedure rather than the reuse of a trained selector. Since uplift depends on the executor, changing the backbone may change which memories are useful. The paper does not evaluate direct scorer transfer or the amount of target-executor feedback needed for adaptation. Successful zero-shot transfer should not be assumed, but characterizing the performance drop and recovery cost would clarify the practical value of maintaining a separate memory-selection module.
Questions for authors
Q1: Broader effectiveness.
Could the authors evaluate the method with a stronger executor or one from a different model family under matched evaluation settings? This would help assess whether the benefits persist as executor capabilities improve.
Q2: Scorer reuse and adaptation.
How well does a scorer trained for one executor transfer to another without additional training? If adaptation is needed, how much feedback from the target executor is required to approach the performance of a scorer adapted specifically to that executor?
6 comments
Response to Q1: Broader Effectiveness with a Different-Family 14B Executor
We thank the reviewer for suggesting a stronger, different-family executor evaluation. We have completed an additional end-to-end comparison with Ministral-14B, extending the main Qwen3-4B/8B experiments to a larger executor from another model family (App. F.9, Table 19). The evaluation covers ALFWorld seen/unseen and BigCodeBench overall/hard, with the same task evaluation protocols and a broad set of memory baselines, including ReasoningBank, MemP, MemRL, Mem0, A-MEM, and LightMem.
Results with Ministral-14B. UpliftMem achieves the highest success rate in all four comparisons. We report ReasoningBank as a directly relevant experiential-memory baseline and the strongest competing method for each metric:
| Success rate (%) | ReasoningBank | Strongest other baseline | UpliftMem | Gain over strongest |
|---|---|---|---|---|
| ALFWorld seen | 60.00 | Mem0 (80.00) | 82.14 | +2.14 pp |
| ALFWorld unseen | 64.18 | Mem0 (72.39) | 74.63 | +2.24 pp |
| BigCodeBench overall | 31.29 | Dense-RAG (38.89) | 40.94 | +2.05 pp |
| BigCodeBench hard | 15.91 | MemRL (27.27) | 29.54 | +2.27 pp |
Compared directly with ReasoningBank, the improvements are +22.14/+10.45 pp on ALFWorld seen/unseen and +9.65/+13.63 pp on BigCodeBench overall/hard. UpliftMem also obtains the fewest ALFWorld unseen execution steps (16.21) among the evaluated methods. The complete results, including execution steps and all baselines, appear in Table 19.
What this establishes. The main Table 1 evaluates two Qwen executors, whereas the new Ministral-14B comparison tests a different model family with more parameters. UpliftMem's leading success rates in both interactive planning and code-generation tasks show that its end-to-end benefits extend beyond the original executor family. This is a separately adapted end-to-end evaluation; direct reuse of a trained scorer without target-side updates is tested independently in our response to Q2.
Updated Manuscript with Stronger-Executor and Zero-Shot Transfer Results
Following your suggestions, we have completed the additional Ministral-14B evaluation and incorporated bidirectional zero-shot scorer-transfer experiments across multiple benchmarks, together with fixed-store and held-out-task-type generalization results. These additions provide direct evidence of UpliftMem's effectiveness beyond the original executor family and the reuse of trained scorers without target-side adaptation. We sincerely appreciate your valuable feedback and would welcome your updated assessment of these new results.
Response to Q2: Zero-Shot Scorer Reuse Across Executors and Task Types
We thank the reviewer for emphasizing scorer portability and target-side adaptation cost. We now report completed bidirectional zero-shot transfer between Qwen3-4B and Qwen3-8B on ALFWorld, WebShop, BigCodeBench, and MemSyco-Bench (Sec. 5.7, Table 4; App. F.8, Table 18), together with leave-one-task-type-out (LOTO) generalization (Table 5; App. E.3). These experiments test whether a trained memory-set scorer can be reused without collecting new target-side feedback or updating its parameters.
Controlled transfer protocols. In Scorer Transfer, the source-trained scorer replaces the target-specific scorer, while the target executor, memory store, and execution protocol remain fixed. This directly isolates reuse of the learned set-value function. Full Transfer additionally reuses the source memory store and protocol. The target executor remains frozen in both settings; neither protocol performs target-side scorer training or memory updates.
End-to-end zero-shot results. The table reports absolute results, combining the matched target-specific configuration (Table 1) with cross-executor changes (App. Table 18). ALFWorld and BigCodeBench entries are success rates (%); WebShop reports reward (0–100) / success rate (%).
With zero additional target-executor feedback, 8B→4B Scorer Transfer improves ALFWorld seen/unseen success by 4.29/4.48 pp over the target-specific scorer; Full Transfer increases these gains to 5.00/7.46 pp and improves WebShop success by 0.40 pp. For 4B→8B, Scorer Transfer matches 79.10% unseen ALFWorld success and improves BigCodeBench hard success by 2.27 pp. These settings already match or exceed executor-specific performance without adaptation.
Fixed-store transfer on MemSyco-Bench. We also test both directions on five identical post-extraction memory stores and five memory-use scenarios, averaged over three independent training runs. The reported metric is the mean absolute out-of-distribution (OOD) versus in-domain (ID) pass-rate gap from each target's own trained scorer (pp; main Table 4), not a gain over native retrieval:
At the default K=5, both transfer directions have a 4.1 pp average absolute gap, decreasing to 1.9/2.8 pp at K=10. This controlled result complements end-to-end success rates by measuring portability of decisions about when and how to use memory.
Completed cross-task generalization. In LOTO, the scorer is trained on four MemSyco task types and evaluated on the excluded fifth, which contributes no warm-up, rollout, probing, or adaptation feedback. Across five stores and three independent training runs, the mean absolute gap from the all-task-trained reference is 13.3, 7.7, and 4.2 pp at K=3,5,10, respectively (Table 5). All LOTO scorers are trained at K=5; only the inference budget changes. At K=5, held-out Objective Fact, Personalized Use, and Valid Selection differ by −0.4, −2.0, and −1.1 pp from the reference.
Adaptation-cost implication. The transfer evaluations establish a zero-target-feedback operating point, rather than merely evaluating a training procedure that must be rerun for every executor. In the ALFWorld and BigCodeBench settings highlighted above, no target-side adaptation is required to reach or exceed the executor-specific reference. Together with the fixed-store and LOTO results, these findings demonstrate reuse of learned set-level valuation across executors and previously unseen task types.