- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper investigates how self-evolving AI agents can decide whether to store acquired knowledge in external memory or integrate it into model parameters. The authors highlight that the better choice depends on factors such as how often the knowledge is used or updated, along with the cost and reliability of each memory mechanism. To address this problem, they propose Dynamic Verified Placement (DVP), which makes placement decisions based on expected long-term benefits, updates these decisions as conditions change, and verifies new knowledge representations before using them. The experiments on LightShop show that external and parametric memory have complementary strengths. Under fixed execution budgets, DVP completes 23.78 and 63.32 more correct tasks than the external-memory baseline on the primary and shifted workloads, respectively. It also improves over the adaptive-amortization baseline by 3.80 and 9.38 correct tasks.
Critical strengths
1- This paper addresses an interesting problem in self-evolving agents: deciding where to store acquired knowledge and when to reconsider previous storage decisions. The idea that external and parametric memory can be beneficial in different situations, depending on how frequently knowledge is reused or updated and the associated execution costs, is interesting and practically relevant.
2- Instead of making a one-time decision, the proposed approach considers the expected future benefits of each memory mechanism and revisits placement decisions as conditions change. Considering execution reliability, verification, and recovery costs when making these decisions is another useful aspect of the framework.
3- The paper evaluates DVP under different workload conditions and compares it with fixed placement, one-step placement, adaptive amortization, and other memory-management strategies. Using matched workloads and executor outcomes makes these comparisons more meaningful. The additional experiments on cold-start behavior, verification, and computational overhead further support the effectiveness of the proposed approach and strengthen the overall evaluation.
Critical weaknesses
1- The paper emphasizes verifying memory representations before using them. However, the main LightShop experiments rely on an exact benchmark verifier that can directly determine whether the resulting task state is correct. In practical agent settings, such verification may not always be available, making it difficult to assess the correctness of an output or action. The authors also provide an evaluation using an independent trace-based verifier, which is useful, but some incorrect outputs still pass verification. Although the paper includes experiments with imperfect verification, it would be helpful to further analyze how DVP performs under more realistic verification conditions, particularly the trade-off between task completion, verification errors, and recovery costs. Such an analysis would provide stronger evidence of the approach's practical applicability.
2- The proposed approach relies heavily on executor reliability estimates, as its placement decisions depend on executor success rates, activation probabilities, and verification risks. However, obtaining reliable estimates for newly acquired knowledge can be challenging, especially when very limited historical information is available. In the main LightShop experiments, the method uses executor-specific information estimated from separate development tasks. Although the authors provide additional cold-start experiments, these also show that collecting such information can be costly. Therefore, it would be useful to further evaluate DVP in more realistic settings where little or no historical information is available, particularly in a zero-shot scenario where the agent continuously encounters new knowledge. This would help demonstrate how effectively DVP can adapt and evolve without relying on previously collected executor information.
3- The proposed approach is mainly evaluated in controlled environments with a limited range of knowledge types, memory mechanisms, and workload conditions. In particular, the LightShop experiments use a small set of predefined rules and version changes, while the ZsRE experiments mainly focus on factual knowledge updates and different external-access costs. However, in real-world scenarios, self-evolving agents may need to handle much larger and more diverse knowledge, including dependencies between stored information, unexpected changes, and interactions between parameter updates. Therefore, I would recommend evaluating the proposed approach in a more complex and realistic agent setting to better demonstrate whether its advantages extend beyond the controlled workloads considered in the paper.
Questions for authors
Please address my all concerns raised in the weaknesses section.