- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
MemEff improves LLM agents with frozen parameters through memory optimization. It measures behavioral efficacy using the difference in action log probabilities with and without memory, then combines this signal with historical reward gains to select and retrieve memories. Across five online rounds on three interactive benchmarks with two execution models, MemEff outperforms the evaluated baselines on average.
Critical strengths
-
The method combines changes in action probabilities with historical reward gains for memory selection and retrieval without updating the execution model's parameters.
-
Table 1 reports average performance gains across three task domains and two execution models. With GPT-5.1, MemEff exceeds the strongest reported baseline on WebArena-Lite by 7.50 percentage points.
Critical weaknesses
-
Changing behavior does not necessarily correct errors. After a failure, Equation 7 selects the memory that most reduces the probability of the original trajectory, while Section 5.5 states that only the first three steps are scored. If the error occurs later, this rule may favor changing a correct prefix. Table 2 measures trajectory deviation but does not verify whether the errors are corrected.
-
The effect of the behavioral efficacy kernel on task performance lacks an ablation. Table 3 compares the full retrieval strategy with semantic retrieval and MemRL but does not isolate the contribution of the behavioral efficacy kernel. Task success rates should be compared with and without this kernel while keeping the candidate pool and other components fixed.
Questions for authors
-
Do the patterns in Figures 1 and 2 hold after controlling for task difficulty and comparing candidate memories for the same task? Specifically, do memories with high rewards but little behavioral impact yield smaller task gains than other candidates?
-
When a failure occurs after the first three steps, how does Equation 7 distinguish changing a correct prefix from correcting the actual error? Can the selection rule be evaluated using subsequent execution outcomes rather than only trajectory similarity?
-
How much does task performance decline when the predictor using both kernels is replaced with the instruction similarity kernel alone, keeping the candidate pool, memory generation, reward gain objective, and fallback rules fixed?