- Rating
- 3(+1)
- Confidence
- 2
- Informativeness
- 3(+1)
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes EquiMem, a memory framework for multi-agent debate (MAD). Specifically, EquiMem identifies a critical issue of previous memory frameworks in MAD: not all memory can augment subsequent reasoning. Therefore, EquiMem formulates memory updating in MAD as a zero-trust memory game and introduces an equilibrium-constrained memory updating principle to ensure all updates apply to the memory and can help task execution. Through evaluation across multiple MAD frameworks, memory architectures, and tasks, EquiMem shows consistent improvements over baselines, which demonstrates the effectiveness.
Critical strengths
- This paper aims to address a critical problems: how to derive a memory that can effectively help subsequent task execution performance. Such direction will make significant contribution to current agentic research, and benefit both academia and industry.
- This paper conducts comprehensive experiments, including 5 kinds of memory architecture, 4 benchmarks, 3 MAD frameworks, and five baselines, which broadly show the advantage of EquiMem.
Critical weaknesses
-
First, one main concern is that the methodology part of this paper is hard to follow. In Section 4.1, I cannot understand why the formulation of calibrated influences is designed as Eq. (3). Besides, the explanation of "How the design reflects the equilibrium objective" is also hard to follow. In general, after reading this part, I cannot understand what kind of memory can be accepted for updating and what kind of memory cannot be accepted. It would be better to improve the writing of the methodology part.
-
Second, this paper seems to have some gaps in each component. (1) Eq. (3) looks like a measurement of an update of memory to task execution. When removing the Nash Equilibrium, Eq. (3) also holds, so it remains unclear what the contribution of the concept of Nash Equilibrium is. (2) This paper applies two kinds of calibration to apply Eq. (3) to memory, but I cannot see the relationship between these two calibrations. Further explanation would be beneficial.
-
Third, this paper uses a relatively small model (Qwen3-VL-8B-Instruct) for experiments, which leaves two kinds of settings under-explored: (1) using more capable models (e.g., GPT5, DeepSeek-V4, and Kimi-K3) separately for experiments and (2) using heterogeneous teams of models, such as using GPT5, DeepSeek-V4, and Kimi-K3 together in one experiment. These are also MAD settings, which this paper does not discuss.
Questions for authors
See critical weaknesses
5 comments
Response, Clarification, and Revisions (2/3)
2. Contribution of Nash equilibrium and relationship between the two calibrations
Weakness #2: Second, this paper seems to have some gaps in each component. (1) Eq. (3) looks like a measurement of an update of memory to task execution. When removing the Nash Equilibrium, Eq. (3) also holds, so it remains unclear what the contribution of the concept of Nash Equilibrium is. (2) This paper applies two kinds of calibration to apply Eq. (3) to memory, but I cannot see the relationship between these two calibrations.
We appreciate the reviewer's questions. We would like to clarify two aspects of our formulation: the role of Nash equilibrium in motivating Eq. (3) and the relationship between embedding-based and graph-based calibration.
-
1) Contribution of the game-theoretic formulation. The zero-trust memory game provides a common principle for determining how memory influence should be regulated in multi-agent debate. As defined in Section 3.1, contributors favor influential updates with low memory conflict, while auditors provide informative memory interactions with low additional effort. This motivates two requirements for memory calibration: an update should remain consistent with the accumulated memory, and its influence should account for information already being used by other agents.
Equation (3) implements these requirements through a conditional optimization. For a fixed update and observed auditor interactions, EquiMem selects the largest feasible influence under the memory-state and memory-use constraints. The running-median tolerances define the operating ranges using recent memory activity.
The game contributes the stability-oriented design principle, and Eq. (3) provides its executable influence-calibration rule. Proposition 3.2 establishes the existence of an equilibrium for the finite game, while Appendix B.2 explains the conditional optimization implemented by EquiMem. These are complementary parts of our formulation.
-
2) Relationship between the two memory calibrations. Embedding-based and graph-based calibration follow the same mathematical formulation in Eq. (3). They compute the same two types of measurements from different memory representations.
For embedding-based memory, d_mem measures excess embedding-neighborhood concentration, while d_use measures changes to auditors' existing retrieval rankings.
For graph-based memory, d_mem measures support from local relations and graph paths, while d_use measures how proposed edges connect to paths already traversed by auditors.
Both methods therefore answer the same questions: how an update fits into the existing memory, and how it affects memory currently used by other agents. Their deviation values are calibrated against separate running-median tolerances, after which the common Eq. (3) determines the influence coefficient. Section 4.4 then applies that coefficient during subsequent retrieval for both memory architectures.
Revision (blue) and highlighted original text (orange).
We have clarified the role of the equilibrium objective in Sections 3.2 and 4.1 and the corresponding conditional optimization in Appendix B.2. We have also expanded the explanation connecting embedding-based and graph-based calibration at the end of Section 4.3, while highlighting their respective deviation definitions in Sections 4.2 and 4.3.
Response, Clarification, and Revisions (3/3)
3. Generalization to stronger LLMs and heterogeneous MAD teams
Weakness #3: Third, this paper uses a relatively small model (Qwen3-VL-8B-Instruct) for experiments, which leaves two kinds of settings under-explored: (1) using more capable models (e.g., GPT5, DeepSeek-V4, and Kimi-K3) separately for experiments and (2) using heterogeneous teams of models, such as using GPT5, DeepSeek-V4, and Kimi-K3 together in one experiment.
We thank the reviewer for highlighting the importance of evaluating EquiMem with stronger LLMs and heterogeneous agent teams. We have expanded Appendix E.5 (Table 5) to cover both settings.
Specifically, the extended evaluation design considers three backbone configurations:
- Homogeneous DeepSeek-V4-Pro: six agents using DeepSeek-V4-Pro.
- Homogeneous Kimi-K3: six agents using Kimi-K3.
- Heterogeneous MAD: six agents comprising two DeepSeek-V4-Pro, two Kimi-K3, and two Qwen3-VL-8B-Instruct agents.
All three configurations use MacNet with six agents, evaluated on both MemBank (embedding-based) and G-Memory (graph-based) across HotpotQA, LoCoMo, ALFWorld, and WebShop. The comparison includes Vanilla, LLM Audit, PPL Filter, A-MemGuard where applicable, MAPLE-Guard, and EquiMem.
Revision (blue).
We have revised Appendix E.5 (Alternative LLM Backbones) and expanded Table 5 to present the larger-backbone and heterogeneous-agent configurations. These additions allow us to examine two important aspects of generalization: the influence of stronger LLM backbones on memory calibration, and the effect of differing agent behaviors and retrieval patterns within a shared-memory system.
The heterogeneous setting is especially relevant because agents using different LLMs may generate different memory updates or access different portions of the same memory. EquiMem uses the shared memory structure and the observed retrieval interactions to calibrate these updates through the same mechanism. Hence, this extension broadens the evaluation setting beyond homogeneous teams of small LLMs and provides a direct framework for examining EquiMem in mixed-backbone MAD systems.
We would like to thank the reviewer once again for their valuable time and thoughtful feedback. We sincerely hope that our responses and revisions have adequately addressed the concerns raised and further improved the clarity and overall quality of the manuscript. We would greatly appreciate the reviewer's further assessment of these improvements and warmly welcome any additional feedback or discussion.
Best,
The author team
Response, Clarification, and Revisions (1/3)
We thank the reviewer for the careful assessment and constructive suggestions. With this opportunity, we have checked the numerical consistency of our experimental results, further clarified the game-theoretic formulation, and strengthened the analysis of statistical significance and memory calibration limitations. Revisions are marked in blue, with relevant existing explanations highlighted in orange. Please see our point-wise responses below.
1. Clarity of Equation (3) and memory-update decisions
We thank the reviewer for this suggestion. We have improved the explanation in Section 4.1 to make the calibration rule and memory-update procedure easier to follow.
The key idea is that EquiMem controls how strongly a proposed update influences future memory retrieval. Every proposed update is first evaluated using two measurements:
Equation (3) then assigns an influence coefficient tau_t \in [0,1]. The two deviation measurements are compared with their respective running-median tolerances, t and t. The rule selects the largest influence that satisfies both constraints.
This gives three intuitive cases. When both deviations are within their permitted ranges, the update receives full influence. When either deviation exceeds its range, the update receives reduced influence. When no eligible auditor interaction exists, calibration uses the memory-deviation constraint alone.
Importantly, EquiMem uses continuous influence calibration. A proposed update is stored with its assigned influence coefficient, and that coefficient controls its contribution to later retrieval. Thus, an unusual update can be retained with limited influence, while a well-supported update can participate fully in subsequent reasoning.
Revision (blue) and highlighted original text (orange).
We have refined Section 4.1, particularly the explanation following Eq. (3), to clarify why the minimum-of-three-terms rule is used and how the resulting coefficient affects memory retrieval. We also highlight the existing definitions of the two deviations in Section 4.1 and the retrieval-weighting mechanism in Section 4.4.