- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
EvoDisco learns a strategy for reorganizing graph-based agent memory while keeping the backbone LLM frozen. A lightweight policy selects an operator, an editing target, and instantiation parameters from a predefined library of Abstract, Refine, Bridge, and Reframe. Learning uses immediate retrieval-quality and complexity feedback, together with a window-level strategic-efficacy signal. Experiments on ALFWorld, WebShop, and LifelongAgentBench OS/DB report improvements over memory baselines, with the largest gains on ALFWorld.
Critical strengths
-
Relevant problem and concrete formulation. Adapting how memory is organized is a worthwhile direction. The paper provides an explicit action space and learning procedure for state-dependent memory editing.
-
Encouraging results and supporting analysis. The ALFWorld improvements are substantial. Component ablations, fixed-operator comparisons, qualitative edits, and policy-transfer experiments help characterize the system beyond aggregate performance.
Critical weaknesses
-
The necessity of the learned controller is not established. A natural alternative is to provide an LLM with the current memory state, previous edits, and observed feedback, and ask it to select the next edit directly. Comparisons against fixed or random selection do not establish that training an additional MLP is preferable to this approach.
-
The scope of strategy discovery is restricted. Operator categories, candidate-generation rules, parameter ranges, and evolution triggers are manually specified. The policy learns choices within this space rather than discovering new operations or learning when to edit. These restrictions may be reasonable, but their benefits and limitations are not sufficiently examined relative to more flexible LLM-directed editing.
-
The fitness proxy requires stronger validation. Immediate utility uses changes in spreading-activation scores and graph complexity rather than downstream task improvement. Higher activation scores do not necessarily imply more useful retrieved information. The paper does not sufficiently establish that this proxy reliably identifies edits that improve task performance.
-
The cost analysis does not justify the architectural choice. End-to-end runtime combines memory maintenance with task execution, and successful episodes can terminate earlier. Consequently, the reported speedups do not isolate controller efficiency. Module-level latency, token usage, and a matched MLP-versus-LLM selector comparison are needed to assess the claimed practical advantage.
-
Limited evidence beyond the evaluated configuration. Experiments use only GPT-5-mini, and gains outside ALFWorld are comparatively small. There are also no comparisons with complete agents using native memory management, such as Claude Code. The results therefore leave uncertainty about the method’s benefits across backbone capabilities and deployed agent systems.
Questions for authors
-
Evidence request: How does the MLP compare with an LLM selector given the same state information, operator library, and historical feedback? This would directly test the controller’s necessity.
-
Clarification: Is there evidence that the retrieval-quality proxy correlates with downstream task improvements, rather than merely increasing graph activation scores?
-
Evidence request: Can the reported cost be separated into task execution, memory writing, operator selection, and edit generation?
-
Scope clarification: What motivates the four-operation library, and which useful memory transformations cannot be represented within it?