- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper investigates the challenge of managing shared memory in long-horizon multi-agent LLM systems. The authors note that the continuous retrieval, writing, and reuse of shared memory by multiple agents leads to memory bloat, the accumulation of redundant information, and increased token overhead. To address this, the authors propose MANGO, a training-free feedback-driven memory management framework. Evaluations conducted across multiple tasks demonstrate significant improvements in memory-token efficiency and resource utilization.
Critical strengths
- The paper addresses the accumulation of redundant or outdated shared memory and its associated token costs challenge in long-horizon multi-agent systems which is practical in real deployment.
- MANGO is a training-free and lightweight design, so its modular design could facilitate integration into existing memory-augmented agent systems.
Critical weaknesses
- The framework heavily relies on agent progress and memory quality , but their computation is unclear. In particular, it is unclear how task progress is estimated and how relevance, novelty, and recency are combined into memory quality scores.
- There are some Inconsistencies in theoretical formulations. Eq. (8) defines governance intensity using , whereas Eq. (22) uses , implying opposite effects of task progress. Similarly, the deletion rule in Eq. (10) depends on , while Appendix H.6 uses . These inconsistencies raise concerns about the validity of the theoretical analysis.
- MANGO improves the proposed efficiency metric but sometimes produces substantially lower reasoning accuracy. For example, on single-objective QA with Qwen3-8B, MANGO achieves an average F1 of 31.00 versus 39.50 for Sliding-window. Accuracy comparisons under matched token budgets are necessary.
Questions for authors
- Can the authors explain how agent progress and memory quality computed in practice?
- Which versions of the governance intensity and deletion rules are actually implemented? Do the theoretical guarantees remain valid after resolving these inconsistencies?
- Can the authors provide accuracy comparisons under matched token budgets to better evaluate the efficiency–accuracy trade-off?
5 comments
Thank you for the detailed response and the additional experiments.
I have one remaining question regarding W1. Could you briefly clarify how task progress and memory quality are computed during actual inference or give an example? In particular, does estimating task progress require ground-truth feedback, or can it be obtained directly from the agents' interactions without external supervision?
Thank you for the valuable question and insight.
We sincerely thank you for the follow-up insights, your great efforts, and are glad the additional experiments were helpful.
Task progress during inference. Each agent maintains a progress score in [0, 1] for its own sub-objective. After every step, the score is updated from three signals: the quality of the agent's answer, the usefulness of the records it retrieved, and the quality of the memory it used. The controller only reads the average score across agents. For example, if an agent answers its sub-question correctly and 3 of its 4 retrieved records are on-topic, its progress increases. Once most agents have made progress, MANGO shifts from admitting new records to consolidating and pruning the shared memory.
Memory quality during inference. Memory quality uses no answer labels. Each record is scored from three signals: its relevance to the current sub-objective, its TF-IDF similarity to the query, and its reliability, determined by the source or agent that wrote it.
Ground-truth feedback is not required. Because the controller only uses a bounded, averaged progress score, the feedback source is interchangeable. In our benchmarks we use task feedback, as in feedback-driven agents such as Reflexion. To verify that MANGO does not depend on it, we replaced this feedback with a fully supervision-free signal, the agreement among agents working on the same sub-objective, keeping everything else unchanged (Qwen3-8B, 8 objectives, 3 seeds).
| MANGO progress signal | F1 | EM | Tokens (×10³) | F1 per 10³ tokens | Memory size |
|---|---|---|---|---|---|
| Task feedback | 44.1 | 22.7 | 30.78 | 1.43 | 22.1 |
| Agent agreement (no supervision) | 44.1 | 22.7 | 30.29 | 1.45 | 21.7 |
Without any ground-truth feedback, MANGO preserves the same answer quality, keeps memory bounded, and uses fewer tokens. This is consistent with our theoretical analysis: governance responds smoothly to progress (Theorem 4), so a different progress estimate leaves its decisions intact.
Incorporated into the revised paper. Motivated by your valuable suggestion, we have incorporated these clarifications into the main text:
- Section 3.2 now introduces that each progress score tracks sub-objective feedback from any source, with the complete progress and quality computation in Appendix B.2 and a phase-by-phase worked example of a MANGO run in Appendix D.1.
- Section 5.5 (Why Closed-Loop Governance Works, Table 5) presents the supervision-free result above, alongside the analysis of why closed-loop governance drives MANGO's gains.
- Appendix H.9 connects this empirical result to the robustness analysis.
We sincerely thank you again for this insightful question, which has strengthened the paper. We have just uploaded the revised version, with all changes highlighted in blue. We would be glad to further discuss any remaining questions or suggestions, and we truly appreciate your time and engagement.
Thank you for your efforts, valuable insights and suggestions.
We sincerely thank the reviewer for recognizing the practical importance of the problem and the lightweight, easy-to-integrate design of MANGO. Motivated by the reviewer's suggestion to compare under matched token budgets, we conducted new budget experiments with all methods sharing the same decoding, injected noise, and seeds. They show that MANGO turns a smaller token budget into more answer quality.
Highlights of the new evidence.
W1 & Q1. Progress and memory quality are lightweight signals read directly from the run, with fixed weights.
Task progress is the task feedback each agent accumulates on its sub-objective (answer quality, retrieval usefulness, memory quality), averaged across agents. Memory quality is a weighted average of three standard signals: the relevance of retrieved records, their TF-IDF similarity to the query, and overall memory reliability, with relevance weighted most. Reliability is derived from each record's provenance (Appendix B.1), and recency decides which records are removed first. The weights are fixed and shared across all experiments. Section 3.3 will state these definitions explicitly.
W2 & Q2. The implementation follows the "remaining demand" design of Eq. (22), so every theoretical guarantee carries over.
Theorem 1 does not depend on the governance intensity. In Theorem 4 and Proposition 1, average progress enters linearly with the same slope magnitude in either notation, so the bounds hold as stated. Deletion is a rounded, per-step-capped count that increases with governance intensity, noise, and occupancy, and Theorem 4 extends to it directly. In practice, memory holds at 22 records at both 150 and 300 steps, exactly as Theorem 1 promotes.
W3 & Q3. MANGO sits on the efficient frontier: no baseline matches its accuracy at its token budget.
Inspired by the your constructive suggestion, we swept every baseline's budget knob toward MANGO's token level:
Even the leanest Retrieval-only setting uses 18.4% more tokens than MANGO and trails it by 6.7 pp in EM. Shrinking a fixed budget does not close the gap either: the grid-tuned Static-Q falls to 12.0 EM. Across all objective counts and seeds, MANGO delivers more F1 per token than the memory baselines in 33 of 36 comparisons. The advantage persists over long horizons:
MANGO wins by spending its budget on cleaner, more relevant memory. For reference, the 31.00 cited from Table 6 is MANGO's EM under Qwen3-8B; its F1 is 35.15, as the two MANGO values appear there in reverse column order. These results will be added to Section 5.3.
We sincerely thank the reviewer again for the constructive and precise comments, and we'll update the revised PDF shortly.