- Rating
- 3(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper introduces MemCon, a learned backend-agnostic wrapper that formulates LLM agent memory management as a sequential decision problem. Memory operations and their parameters are selected based on task progress and memory state. The policy is learned online using a tabular contextual bandit with UCB exploration, initialized with manually-specified heuristic priors and updated from task-level feedback. The authors evaluate MemCon across six benchmarks, three agent frameworks, and three LLM backbones and show improvements in task success and token efficiency over baselines.
Critical strengths
The paper’s strengths are (1) its generalizable approach to adaptive agent memory management, and (2) its comprehensive empirical evaluation and ablation studies showing the controller’s contribution.
First, MemCon’s backend-agnostic controller learns its policy online without additional LLM training or inference for policy selection. It addresses the limitations of fixed memory access methods that may either retrieve information too aggressively or too conservatively. Learning memory operations is not new, but MemCon provides a lightweight approach that considers both task progress and memory status to adaptively select memory operations and parameters.
Second, the authors conduct extensive comparisons against memory control baselines across six benchmarks, three agent frameworks, and three LLM backbones. These experiments show MemCon’s improvement in task success and token efficiency. The component ablation on ALFWorld shows that the learned controller alone improves performance from 59.7% to 64.9%, which accounts for most of the system’s improvement.
Critical weaknesses
Although the authors show improvements over fixed and several adaptive memory access methods, they do not sufficiently compare against simpler alternatives. For example, the authors could try epsilon-greedy exploration with the same states, actions, and Q-value updates. Or try a non-learning, state-dependent heuristic policy using the same states and actions.
I also found the paper’s figures difficult to parse because they were either crowded with text or had small fonts.
Questions for authors
Could the authors please compare their method against simpler controllers, such as epsilon-greedy exploration or a non-learning, state-dependent heuristic policy using the same states and actions?
Emergency Review from AC (Paper was desk rejected, but got reverted later, therefore there were only 2 assigned reviewers)
The paper proposes a lightweight online controller for agent memory. I like the practical motivation: retrieval should adapt to the task and the available experience. The gains in both success rate and input-token use are encouraging, and the ablation shows that learned control contributes beyond the added planning heuristics.
My main reservation is that the method is fairly incremental, combining tabular UCB with hand-designed states and actions. Most main results also come from single runs, so the smaller improvements are hard to assess. The MDP framing is broader than what the implemented credit assignment supports, particularly for maintenance whose benefits appear in later tasks.
Overall, I lean toward weak accept. The contribution is useful and reasonably well supported, though stronger evidence across repeated runs would make the case more convincing.