- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
This paper introduces SOMC, which formulates long-term LLM memory compression as submodular optimization under a token budget. It selects existing memory chunks based on coverage, diversity, and query relevance, avoiding repeated LLM-based summarization. The authors develop algorithms for both full-history access and one-pass streaming, with approximation guarantees that depend on chunk granularity. Experiments on LoCoMo and LongMemEval show that SOMC reduces memory size and management costs while maintaining competitive answer quality.
Critical strengths
- Selecting original memory chunks avoids repeated LLM-based summarization. The approach is practical and achieves competitive results on two benchmarks.
- The paper covers both full-history and streaming access, with approximation guarantees that explicitly account for chunk size.
Critical weaknesses
- The theoretical guarantees apply to the submodular objective, not directly to answer quality. Coverage and diversity may not adequately capture the value of evidence that requires reasoning across multiple chunks.
- The streaming algorithm maintains multiple candidate sets, so the output budget does not reflect its actual working memory. Peak memory usage, including candidate sets, embeddings, and caches, is not reported.
- The static and streaming variants use different coverage objectives, so their performance gap cannot be attributed solely to the restrictions of streaming access.
Questions for authors
- In Table 6, replacing STATIC-SMK with density-based greedy under the same objective and budget drops the LoCoMo score from 74.6 to 41.5. Why is the gap so large? Could the authors report objective values, actual token usage, and evidence recall to help explain it?
- How much data does STREAMING-SMK actually retain during execution? Does it still offer an advantage when all candidate sets and auxiliary state count toward the memory budget?
- Could the static and streaming algorithms be compared using the same coverage objective? This would help separate the cost of not seeing future information from the effect of changing the objective.