- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper proposes DecentMem, a decentralized memory framework for LLM-based multi-agent systems. Each agent maintains an exploitation pool for retrieving historical experiences and an exploration pool for generating new experiences. An online router dynamically balances the two based on stage-wise feedback from an LLM judge. The paper provides theoretical guarantees on global reachability and logarithmic regret, and evaluates the framework across multiple MAS architectures, LLM backbones, and benchmarks.
Critical strengths
-
Meaningful problem motivation. The paper addresses an interesting problem of memory management in multi-agent systems. Maintaining agent-specific experiences rather than a shared memory repository is a reasonable approach to preserving role specialization and reducing unnecessary memory retrieval and coordination overhead.
-
Extensive empirical evaluation. The experiments cover three MAS frameworks, five LLM backbones, and diverse benchmarks. The reported improvements in task performance, token efficiency, and behavioral diversity are promising. The additional ablations on routing strategies and memory ownership also provide useful evidence for the proposed design.
Critical weaknesses
-
Limited novelty and insufficient justification of the memory design. The distinction between the exploitation and exploration pools appears somewhat artificial. According to Appendix F, exploration essentially corresponds to solving a task without retrieving historical memory, making the framework resemble an adaptive retrieval or context-selection mechanism rather than a fundamentally new memory architecture. It is unclear why two separate memory pools are necessary instead of a single memory repository with an adaptive retrieval policy. More importantly, the practical memory management mechanisms are insufficiently explained: How are memory prototypes constructed and consolidated? How are conflicting, redundant, or outdated memories handled as the repository grows? How exactly do retrieved collaboration trajectories influence agent coordination beyond prompt injection? Similarly, although behavioral diversity is measured, it remains unclear whether the improvement comes from decentralized memory itself or from role-specific prompting and other design choices.
-
Insufficient baselines and questionable connection between theory and implementation. The evaluation lacks essential single-agent and compute-matched Best-of-N or self-consistency baselines, making it difficult to establish the benefits of multi-agent collaboration over simpler inference strategies. In particular, the reported Qwen3-4B performance on AIME24/25 appears unusually low and warrants clarification of the inference configuration, including thinking mode, generation budget, and evaluation protocol. Comparisons with simpler memory retrieval methods and standard bandit algorithms (e.g., UCB) would also help establish the value of the proposed router. Furthermore, the theoretical regret guarantee relies on strong assumptions about the reward function and routing dynamics, whose validity in the actual implementation is unclear. The stage-wise feedback compares scores from consecutive collaboration stages, which may have different objectives, raising concerns about whether the resulting updates reliably reflect the effectiveness of the selected memory pool. Overall, the theoretical analysis seems insufficiently connected to the practical algorithm.
Questions for authors
See Critical Weaknesses
4 comments
Continued from the previous comment
W2
The evaluation lacks essential single-agent and compute-matched Best-of-N or self-consistency baselines including thinking mode, generation budget, and evaluation protocol.
We thank the reviewer for raising this point. We additionally evaluate Gemma4-E2B under single-agent and self-consistency settings and compare them with DecentMem using the same backbone. DecentMem consistently outperforms both baselines. We also note that the no-memory DyLAN setting can sometimes perform worse than the single-agent baseline, as dynamic multi-agent coordination without persistent memory or feedback from the LLM-as-a-judge can lead to less effective cooperation.
| Framework | Method | AIME25&24 | MBPP-Plus | ALFWorld | BBH |
|---|---|---|---|---|---|
| DyLAN | Single agent | 21.67 | 54.62 | 57.93 | 32.17 |
| Self-consistency N=9 | 25.00 | 58.13 | 63.17 | 38.63 | |
| DecentMem | 28.33 | 66.79 | 69.53 | 43.47 |
We thank the reviewer for pointing this out. We have added more details on the inference and evaluation settings for reproducibility. All Qwen3 and Gemma4 models are evaluated in non-thinking mode (enable_thinking=False). For generation, we follow the recommended decoding configurations for each model family: for Qwen3, we use temperature (=0.7), top-(p=0.8), and top-(k=20); for Gemma4, we use temperature (=1.0), top-(p=0.95), and top-(k=64).
For the generation budget, we use the same maximum output length of 32798 tokens for each model generation across all compared memory methods under the same backbone and MAS framework. The prompts used for task solving, decomposition, and agent interaction are already provided in the Appendix.
For evaluation, Qwen3 models are run locally through Ollama and Gemma4 models through Hugging Face Transformers. All memory methods and the no-memory baseline use the same backbone, benchmark examples, generation settings, and evaluation procedure within each experimental setting. We evaluate performance using the task-specific success metric, primarily accuracy, and use the same Qwen-32B LLM-as-a-judge across both Qwen- and Gemma-based agents for stage-wise feedback.
Comparisons with simpler memory retrieval methods and standard bandit algorithms (e.g., UCB) would also help establish the value of the proposed router.
The simpler memory retrieval baseline corresponds to the exploitation-only setting in Table 3. We further compare our router with UCB, and the results are shown in the following table. The overall trend remains consistent with the results reported in the draft. In particular, the online router can better adapt to different task preferences between exploitation and exploration, making it more effective across different tasks.
| Routing | Qwen3-8B BBH | Qwen3-8B AIME | Gemma4-E4B BBH | Gemma4-E4B AIME |
|---|---|---|---|---|
| Exploit | 74.27 | 26.67 | 57.23 | 30.00 |
| UCB | 66.13 | 30.00 | 55.23 | 38.33 |
| Online | 76.19 | 36.67 | 59.69 | 43.33 |
Furthermore, the theoretical regret guarantee relies on strong assumptions about the reward function and routing dynamics, whose validity in the actual implementation is unclear. The stage-wise feedback compares scores from consecutive collaboration stages, which may have different objectives, raising concerns about whether the resulting updates reliably reflect the effectiveness of the selected memory pool. Overall, the theoretical analysis seems insufficiently connected to the practical algorithm.
We thank the reviewer for raising this point. We will clarify that the regret result is a conditional guarantee under the stated assumptions, with Assumption 2 requiring the routing fixed point to coincide with the optimum. Our stochastic-approximation analysis already allows bounded zero-mean fluctuations through the martingale-difference noise term. Regarding stronger systematic or highly non-stationary deviations, we will acknowledge them as a limitation of the current analysis and an important direction for future work.
Thank you for the detailed response and additional experiments. The new single-agent, self-consistency, and UCB comparisons, along with the clarified implementation details, address several of my concerns.
However, I remain somewhat concerned about the novelty of the dual-pool design compared to simpler adaptive retrieval strategies, as well as the gap between the theoretical assumptions and the practical routing algorithm. In particular, the reliability of comparing feedback across different collaboration stages remains unclear.
- Novelty of the dual-pool design compared to simpler adaptive retrieval strategies
| Routing | Qwen3-8B BBH | Qwen3-8B AIME |
|---|---|---|
| Single Adaptive Retrieval | 73.29 | 31.67 |
| DecentMem | 76.19 | 36.67 |
We thank the reviewer for raising this point. Additionally, we conduct a comparison between DecentMem and Single Adaptive Retrieval using Qwen3-8B under the AgentNet framework. In Single Adaptive Retrieval, each agent maintains a single memory pool and first retrieves relevant historical memories. If the retrieval similarity falls below the threshold of 0.8, following the same setting as DecentMem, the agent generates a new memory piece for exploration. As the results show, DecentMem consistently outperforms Single Adaptive Retrieval. Notably, the performance gap is larger on the mathematical reasoning benchmark (AIME) than on BBH. One possible explanation is that mathematical reasoning tasks may require more exploration of alternative solution strategies, whereas Single Adaptive Retrieval can still be influenced by previously retrieved memories through the Transformer's attention mechanism, potentially constraining its ability to explore novel solutions. In contrast, DecentMem explicitly separates exploitation and exploration, allowing the exploration branch to generate new proposals without conditioning on previously retrieved historical memories.
From an engineering perspective, with a single adaptive retrieval or context-selection mechanism, even if an agent decides to generate a new proposal after searching the memory pool, its generation may still be partially influenced by previously retrieved information. This is because the retrieved memory has already entered the model's context and can affect subsequent token generation through the Transformer's attention mechanism.
By explicitly separating the two memory pools, DecentMem implements a mechanism similar to a two-stage decision system. In the first stage, the agent searches the exploitation pool to identify relevant historical experiences. If no useful memory is found, it proceeds to the second stage, where the exploration branch operates with a fresh context that excludes previously retrieved memories, allowing it to generate genuinely new proposals without being influenced by the historical information considered in the first stage. Hence, DecentMem separates not only memory storage and retrieval policies, but also the information available to the model during exploitation and exploration.
- The gap between the theoretical assumptions and the practical routing algorithm
To further address the gap between the theoretical assumptions and the practical routing algorithm, we have made two revisions.
First, we explicitly incorporate the actual initialization into our theoretical analysis by introducing a localization condition, allowing the convergence analysis to account for the practical starting point.
Second, regarding the assumptions on the reward function, the stage-wise reward is obtained from LLM-as-a-judge feedback, while and denote the corresponding expected rewards under each routing probability. Since these reward functions are induced by the underlying LLM and task distribution and are not available in closed form, we clarify that the theoretical convergence guarantee holds if the reward functions and the resulting routing dynamics satisfy the stated stability and localization assumptions. As these conditions cannot be guaranteed for arbitrary LLM-induced reward functions, we explicitly acknowledge this as a limitation in the revised version.
The corresponding assumptions and detailed proofs are provided in Appendix A.2 (Assumption 2 and Theorem 4, pp. 15–18) of the latest revised version.
- The reliability of comparing feedback across different collaboration stages
| Framework | Method | AIME25&24 |
|---|---|---|
| AgentNet | DecentMem | 23.33 |
| 10% | 21.67 | |
| First stage 10% | 23.33 | |
| Second stage 10% | 20.00 | |
| Third Stage 10% | 20.00 |
We thank the reviewer for raising this point. We further conduct a robustness test by introducing 10% incorrect stage-level scores from the LLM-as-a-judge, considering all stages together as well as the first, second, and third stages separately, using Qwen3-4B under the AgentNet framework. As the results show, intermediate reasoning stages play an important role in mathematical reasoning tasks, as incorrect feedback at the middle stage has a greater impact on final accuracy. Overall, the results demonstrate that DecentMem remains relatively robust to incorrect stage-level feedback.
W1
We thank the reviewer for this point. The two pools are intended to implement two distinct search policies: the E-pool proposes actions conditioned on historically relevant experience, whereas the X-pool generates fresh, history-independent proposals from the LLM prior. As illustrated by the global-reachability analysis, the former performs a similarity-based local walk, while the latter provides a teleportation mechanism that can reach regions not induced by retrieved experience. With a single adaptive retrieval or context-selection mechanism, even if the system decides to propose a new action after searching the shared repository, that proposal can still be partly influenced by the retrieved relevant information. By explicitly separating the two pools, DecentMem allows the exploration branch to generate genuinely new proposals without conditioning on historical retrieval.
Additionally, G-Memory is an example of a single adaptive retrieval method. We conduct extensive experiments comparing DecentMem with G-Memory, and the results show that DecentMem outperforms G-Memory, further supporting the benefit of explicitly separating exploitation and exploration.
We thank the reviewer for raising these implementation details. Each memory piece is represented as z=(ξ,r⋆), where ξ is the context prototype and r⋆=(rtrajectory,rcomment) records the action/cooperation trajectory together with the agent’s rationale. New experiences are first stored in the X-pool and, after the task finishes, consolidated into the E-pool.
As the repository grows, the E-pool is compressed and capped at 100 memory prototypes rather than growing indefinitely. Redundant or overlapping experiences are reduced through this compression. During retrieval, we use Top-K similarity with a threshold. If no sufficiently relevant memory is found, the router falls back to exploration. We will make these implementation details clearer in the revision.
For coordination, the stored trajectory includes task decomposition, cooperation history, and which agent handled each sub-task. When a compatible trajectory is retrieved, its previous allocation can be reused as a coordination prior for the next stage. In the current implementation, this information is passed through the agent context.
We thank the reviewer for this point. G-Memory serves as a centralized-memory baseline under the same role-specific agent setup, which helps control for the effects of role-specific prompting and retrieval. Since DecentMem still outperforms G-Memory in most comparisons, the results suggest that the improvement cannot be explained by role-specific prompting alone and is associated with the decentralized memory design.