- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes MemFolder, which combines record-level consolidation (RecordFolder) and context-level compression (ContextFolder) to improve agent memory efficiency. Experiments on text-based and multimodal benchmarks show promising results, including 99.0% performance retention with 50% fewer records on Mem0. However, the novelty over existing memory compression methods, the reliability of factual verification, and the end-to-end efficiency gains remain insufficiently established.
Critical strengths
- The paper addresses two distinct bottlenecks in agent memory management: the number of stored records and the length of retrieved contexts. Treating them separately is a sensible design choice, particularly for agents that need to maintain long interaction histories under limited memory and context budgets.
- RecordFolder and ContextFolder target different sources of memory overhead, making their combination more comprehensive than record pruning or context compression alone. The evaluation across text-based and multimodal settings also provides initial evidence that substantial memory reduction is possible without a proportional drop in downstream performance.
Critical weaknesses
- The novelty over existing memory compression methods remains unclear. MemForest, MemRefine, and StreamMeCo have explored related directions, including memory consolidation and compression [1–3]. Although MemFolder combines record-level and context-level compression, the methodological advance beyond these approaches is not sufficiently established. The relatively small contribution of some components in the ablations further weakens the novelty claim.
- The evidence for factual reliability and practical efficiency is incomplete. ContextFolder relies on an LLM-based verification loop, yet the paper acknowledges inconsistent verifier judgments, and facts omitted during extraction may never be checked. In addition, the reported speedups mainly concern retrieval, while preprocessing also incurs LLM and verification costs. Consequently, neither reliable fact preservation nor the overall deployment benefits are fully established. [1] Wang et al. MemForest: Efficient Agent Memory Management via EventTree Partitioning and Progressive Merging. arXiv:2609.08273, 2026. [2] Kim et al. MemRefine: LLM-Guided Compression for Long-Term Agent Memory. arXiv:2606.13177, 2026. [3] Wang et al. StreamMeCo: Long-Term Agent Memory Compression for Efficient Streaming Video Understanding. Findings of ACL 2026.
Questions for authors
- A controlled comparison with MemForest and MemRefine under comparable memory budgets would help clarify the methodological differences and the additional value of MemFolder.
- Reporting whether preprocessing costs, including LLM calls and factual verification, are included in the latency evaluation would provide a clearer picture of the overall efficiency gains.
- An independent factual evaluation, including cases where important facts are omitted or incorrectly preserved, would provide stronger evidence of the reliability of ContextFolder.
7 comments
Response to Reviewer BjJm (2/5: Weakness 1 - Ablations & Question 1)
Weakness 1 (Continued): Component Ablation Analysis
The ablation results in Tables 4–7 and Figure 5B reflect proximity to the uncompressed baseline and targeted gains on cross-session, preference, and knowledge-update subtasks:
- 1. Structured grouping over standard clustering (Tables 4 & 5): At 50% record reduction, DPC achieves the highest average score among tested anchor strategies (57.2, beating KMeans 55.4 by +1.8 and runner-up MinMax 56.8 by +0.4, retaining 99.1% of the uncompressed 57.7 baseline), while removing cosine similarity drops accuracy by 3.4 points. Operating near the uncompressed baseline, gain gating (+1.9), entity affinity (+1.1), and temporal chaining (+0.6) act as fine-grained structural refinements.
- 2. Substantial gains on targeted LongMemEval subtasks (Tables 1 & 2): Overall averages are diluted by single-hop questions requiring one record. On subtasks directly testing cross-session entity resolution and temporal state updates, RecordFolder achieves pronounced gains over MemForest consistent with our design motivation: in the full record-compression evaluation on Mem0 (Table 1, 50% reduction), it leads overall by +3.2 points (55.0 vs. 51.8; +5.3 on Multi-Session MS, +10.0 on Preference SP, +3.0 on Temporal TR) and by +30.0 points on SP at 70% reduction; on MemOS (Table 2, 50% reduction), it leads by +16.6 points on SP (83.3 vs. 66.7), +10.2 points on Knowledge Update KU (75.6 vs. 65.4), and +4.5 points on MS (45.1 vs. 40.6).
- 3. Clear value of planning and blind verification (Table 7 & Figure 5B): Removing Planner increases unit length by +32.0% (12.31 -> 16.25 tokens on LoCoMo) and reduces accuracy by 0.7–0.8 points; removing Verifier drops accuracy by 1.9 points (LoCoMo) and 1.4 points (LongMemEval) at slightly lower achieved tokens (11.94 vs. 12.31). Increasing revision iterations from 1 to 3 (Figure 5B) boosts accuracy by +2.14 points on LoCoMo (54.42% -> 56.56%) and +3.00 points on LongMemEval (48.80% -> 51.80%) at nearly identical token counts.
Question 1: Controlled Comparison Under Comparable Memory Budgets
In the context-compression evaluation at 50% record reduction (Figure 3, tabulated below), full MemFolder operates in a compact ~10.8–13.4 tokens/record regime, outperforming MemForest+ContextFolder by +2.8 points on LoCoMo and LLMLingua-2 by up to +12.2 points, while ContextFolder cuts MemForest and ToMe outputs by 23.2%–42.6%:
| Record Method (Keep 50%) | Context Method | LoCoMo Tok/Rec | LoCoMo ACC | LongMemEval Tok/Rec | LongMemEval ACC | M3-Robot Tok/Rec | M3-Robot ACC |
|---|---|---|---|---|---|---|---|
| RecordFolder (Ours) | Vanilla | 20.60 | 59.3 | 21.60 | 55.2 | 30.2 | 31.0 |
| + BabelTele | 27.30 (+32.5%) | 49.3 | 17.20 (-20.4%) | 45.8 | 30.2 (0.0%) | 32.1 | |
| + LLMLingua-2 | 10.80 (-47.6%) | 44.4 | 12.00 (-44.4%) | 47.8 | 31.7 (+5.0%) | 29.5 | |
| + ContextFolder (Ours) | 12.31 (-40.2%) | 56.6 | 13.40 (-38.0%) | 51.8 | 20.9 (-30.8%) | 29.5 | |
| MemForest | Vanilla | 19.10 | 58.6 | 20.80 | 54.6 | 30.4 | 31.6 |
| + BabelTele | 27.00 (+41.4%) | 48.4 | 16.90 (-18.8%) | 45.0 | 29.3 (-3.6%) | 31.5 | |
| + LLMLingua-2 | 10.20 (-46.6%) | 42.5 | 11.40 (-45.2%) | 48.2 | 31.8 (+4.6%) | 29.9 | |
| + ContextFolder (Ours) | 11.50 (-39.8%) | 53.8 | 13.30 (-36.1%) | 52.2 | 20.7 (-31.9%) | 27.4 | |
| ToMe | Vanilla | 22.30 | 55.7 | 19.80 | 52.6 | 31.7 | 30.9 |
| + BabelTele | 30.00 (+34.5%) | 46.5 | 17.10 (-13.6%) | 47.8 | 30.6 (-3.5%) | 30.6 | |
| + LLMLingua-2 | 11.90 (-46.6%) | 36.9 | 11.00 (-44.4%) | 47.4 | 31.8 (+0.3%) | 31.4 | |
| + ContextFolder (Ours) | 12.80 (-42.6%) | 51.8 | 15.20 (-23.2%) | 51.2 | 21.5 (-32.2%) | 29.0 |
- Controlled comparison with MemRefine's mechanism: MemRefine [2] operates on multi-field structured notes (keywords, tags, context descriptions) under a coupled JSON-serialized byte budget using local pairwise {delete, merge, preserve} actions without a dedicated intra-group syntax encoder. Under identical anchors and storage budgets on our conversational stores, Table 6 (Merging 57.2 vs. Pruning 51.1, a +6.1-point advantage) and Figure 3 above (where adding verified compact encoding cuts unit tokens by 31%–40% while outperforming token-deletion baselines by up to +12.2 points) directly isolate and confirm the advantage of MemFolder's decoupled consolidation and verified encoding over deletion-reliant compression.
Response to Reviewer BjJm (3/5: Weakness 2 - Factual Reliability)
Weakness 2 (Part 1): ContextFolder Factual Reliability, Trace Audit, and Held-Out Utility
While we explicitly frame ContextFolder's verification loop as an asymmetric failure-detection heuristic rather than a formal lossless guarantee (Section 3.4), dual-input access to source records, reference-aligned judging, and length-safe fallbacks measurably protect factual fidelity in both source-level production traces and held-out QA evaluations.
1. Defense Mechanisms and Residual Boundaries Across Four Factual Failure Modes (Section 3.4 & Figures 9–14)
| Failure Mode | Reviewer's Concern | Built-in Safeguard in ContextFolder | Observed Behavior & Residual Boundary |
|---|---|---|---|
| 1. Extraction Omission | Facts omitted by Planner are never checked by Verifier | Dual-input Compressor (Fig. 10): Receives both {facts_list} and complete raw group records {lines}. Adaptive density extraction (Fig. 9, ~1 fact/10 tokens, up to 12) prioritizes high-tier facts. | Prevents Planner omissions from cutting off source access; verification coverage remains bounded by extracted checks, and low-priority modifiers in >12-fact groups are unchecked. |
| 2. Compression Omission | Compressor drops dates, negations, or key facts for token budget | Tiered retention (Fig. 10): Mandates non-inferable high-tier facts (and med-tier when tau >= 0.65) with exact names/dates/numbers. Blind reader check (Fig. 11): Forces "UNKNOWN" when missing, triggering MUST-FIX revision (Fig. 13). | Prioritizes shortest passing candidate; if none passes in 3 rounds, selects by weighted recovery (3:2:1) and falls back to a shorter candidate (below S_G) or raw concatenation if length >= S_G. |
| 3. Substitution Error | Blind reader answers a missing fact using another fact in the encoding | Reference-aligned Fact Judge (Fig. 12): Compares blind answers against Expected Answers, requiring exact numbers, dates, and entity names; mismatches fail and trigger revision. | Catches "same-category fact substitution" that a simple "UNKNOWN" check would miss (Case C below). |
| 4. Verifier Inconsistency | Same encoding receives inconsistent judgments across rounds (App. B.4) | Asymmetric isolation (App. B.4): Blind reader isolation prevents direct answer copying from source text; conservative "UNKNOWN" outputs prompt clearer phrasing in subsequent rounds. | While Fact Judge's lenient semantic matching does not provide a zero-false-pass formal guarantee, multi-round revision and fallback protection substantially reduce missed facts in practice. |
2. Source-Level Trace Audit (Appendix B.1–B.3) Across the three unmodified LoCoMo production traces in Appendix B (22 source records totaling 202 tokens -> 88 tokens, -56.4%), 13/13 Planner-extracted checks passed within 1–3 rounds:
- Intercepting Fact Substitution (App. B.3, Case C): In a 12-record group (92 tokens), Round 1 (44 tokens) omitted Fact 3 ("Values counseling and support groups"), and the blind reader substituted Fact 6 ("sharing stories and giving others hope"). Fact Judge caught this mismatch against the expected answer and flagged Fact 3 for MUST-FIX; Round 2 (50 tokens, 0.54 ratio) restored it and passed all 6 checks.
- Resolving Coreference Ambiguity (App. B.1, Case A): In a 4-record group (48 tokens), Rounds 1 (19 tokens) and 2 (26 tokens) failed Question 4 ("UNKNOWN"); Round 3 (21 tokens, 0.44 ratio) passed all 4 checks (weighted recovery 1.0) and was selected over the shorter incomplete Round 1.
- Single-Round High-Redundancy Folding (App. B.2, Case B): In a 6-record group (62 tokens), Round 1 passed all 3 checks in 17 tokens (0.27 ratio, -72.6% tokens).
3. Held-Out Downstream QA Ablations (Zero Test-QA Input)
| Method | Planner | Verifier | LoCoMo Tok/Rec | LoCoMo ACC (%) | LongMemEval Tok/Rec | LongMemEval ACC (%) |
|---|---|---|---|---|---|---|
| RecordFolder + LLMLingua-2 | No | No | 10.80 | 44.4 (-12.2) | 12.00 | 47.8 (-4.0) |
| RecordFolder + BabelTele | No | No | 27.30 | 49.3 (-7.3) | 17.20 | 45.8 (-6.0) |
| ContextFolder (w/o planner, Table 7) | No | Yes (3 rounds) | 16.25 | 55.9 (-0.7) | 14.44 | 51.0 (-0.8) |
| ContextFolder (w/o verifier, Table 7) | Yes | No (0 rounds) | 11.94 | 54.7 (-1.9) | 13.19 | 50.4 (-1.4) |
| ContextFolder (Iteration = 1, Fig. 5B) | Yes | 1 pass (no rev.) | 11.94 | 54.42 (-2.14) | 13.08 | 48.80 (-3.00) |
| Full ContextFolder (Ours, 3 rounds) | Yes | Yes (up to 3) | 12.31 | 56.56 | 13.40 | 51.80 |
At comparable achieved token lengths (LoCoMo: 11.94–12.31; LongMemEval: 13.08–13.40 tokens/record), 3-round verification improves downstream accuracy by +1.9 and +1.4 points over w/o verifier, and by +2.14 and +3.00 points over single-pass Iteration = 1.
Response to Reviewer BjJm (1/5: Weakness 1 - Methodological Advance)
Thank you very much for your constructive review and for recognizing that decoupling stored record count from retrieved context length is a sensible design choice, that RecordFolder and ContextFolder target complementary overhead sources more comprehensively than pruning or context compression alone, and that our text and multimodal evaluations demonstrate substantial memory reduction without a proportional performance drop. Below, we respond to each of your weaknesses and questions in detail:
Weakness 1: Methodological Advance Over Existing Compression Methods (Part 1: Mechanisms & Macro-Average Results)
Unlike existing methods that address a single bottleneck—either losing evidence via pruning or inflating unit length by 2.12x–3.87x via merging—MemFolder introduces a reusable shared grouping structure that decouples and sequentially optimizes record count N and per-unit length l.
1. Architectural Comparison with StreamMeCo, MemRefine, and MemForest
2. Macro-Average Accuracy Under Fixed Record-Count Budgets (Tables 1–3 Summary) Within each table and figure, all methods are evaluated on identical memory snapshots, retrievers (fixed top-k), and QA protocols. RecordFolder achieves the highest macro-average accuracy across every evaluated framework and reduction ratio:
In Part 2/5, we address component ablations and Question 1's dual-axis budget comparisons.