- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes FiniteMem, a finite-capacity memory management framework for LLM-based multi-agent systems (MAS), aiming to improve task performance while reducing storage, token consumption, and runtime overhead. The framework introduces a three-layer memory architecture consisting of trajectory, knowledge, and perspicacity layers, providing both fine-grained action guidance and high-level reasoning support. It further develops two memory replay strategies based on memory competition and inhibitory control to selectively retain valuable information under capacity constraints. The authors evaluate FiniteMem across five benchmarks and three multi-agent frameworks (AutoGen, DyLAN, and MacNet), reporting consistent performance improvements over existing memory methods, alongside reductions of 31.3% in storage, 26.9% in token consumption, and 11.8% in runtime compared with G-Memory. Additional ablation studies, sensitivity analyses, and experiments with smaller language models further support the effectiveness of the proposed approach.
Critical strengths
-
FiniteMem addresses the important challenge of managing growing memory in long-running multi-agent systems. It achieves competitive or better task performance while reducing storage, token consumption, and runtime, demonstrating practical value beyond accuracy improvements alone.
-
The method is evaluated across five benchmarks and three multi-agent frameworks, with additional experiments using different model sizes. The ablation studies and sensitivity analyses further support the effectiveness of the proposed memory architecture and replay strategies, suggesting that the approach can generalize across different settings.
Critical weaknesses
-
The paper compares FiniteMem mainly against methods without hard memory capacity constraints, but lacks comparisons with simpler eviction strategies under the same memory budget. Moreover, the main experimental capacity limits scale with dataset size, leaving the claimed constant-space advantage insufficiently established. These issues make it difficult to isolate the benefits of the proposed replay algorithms.
-
Appendix B.3 describes an AutoGen configuration in which a Ground Truth Agent checks outputs against standard answers. Are these answers accessible during evaluation or only used for feedback on completed tasks? This distinction is important because access to ground-truth answers during evaluation could compromise the validity of the reported results.
Questions for authors
-
Appendix B.3 states that the Ground Truth Agent checks outputs against standard answers, while Section 3.1 indicates that only correctly answered trajectories are stored. Could the authors clarify when and how these correctness signals are obtained? In particular, are ground-truth answers accessible during test-time execution, or only after a task has been completed? This distinction is important for assessing the validity of the evaluation.
-
Could the authors compare FiniteMem with simpler memory retention strategies, such as FIFO, random eviction, or recency-based selection, under the same memory capacity? Such comparisons would help establish whether the improvements come from the proposed replay mechanisms rather than memory capacity constraints alone.
-
The main experiments set memory capacities proportional to dataset size, while the paper emphasizes constant storage complexity. Could the authors clarify how these capacity limits would behave in an open-ended task stream where the total number of future tasks is unknown?