- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
- This paper studies multi-source memory fusion for agents whose evidence is fragmented across devices, users, and time.
- It introduces MemFuseBench, a fully synthetic benchmark containing 7,823 events and 357 questions across six scenarios and six diagnostic categories. A Scene-to-Sensor pipeline constructs personas and causal storylines, projects them into source-specific observations, and generates evidence-grounded questions, answer checklists, and distractors, with reviewer-corrector validation.
- The paper also proposes MemFuse, a training-free memory system combining immutable source-tagged atomic events, fused cluster memories, and membership, causal, and semantic graph relations. LLM agents organize incoming events and perform iterative constrained retrieval; the final answer is generated from 20 retrieved atomic events.
- Across three LLM settings, MemFuse achieves the highest observed overall checklist coverage, although it does not consistently outperform the full-context reference. The ablations indicate that iterative retrieval and retrieval constraints account for the largest overall gains.
Critical strengths
- The benchmark targets a meaningful and useful memory problem. Complementary observations, source conflicts, and different user perspectives provide a focused setting for studying evidence integration beyond recalling facts from a conversational history. Explicit evidence-event identifiers and answer checklists make the task more auditable and potentially useful for diagnosing retrieval failures.
- The method preserves original evidence while using fused summaries as retrieval abstractions. This separation is a sensible design for maintaining provenance during memory aggregation. Experiments across three LLM settings show substantial improvements over naive RAG, and the component ablations and token-usage reporting help characterize both effectiveness and cost.
Critical weaknesses
- It looks like wo/ fusion doesn't drop much performance. Not sure if it is a useful mechanism, especially if the author uses
museas the main story. - Unclear compute-performance tradeoff. A shared top-20 item limit does not equalize context length or internal LLM/search computation. Table 2 reports roughly 9-11 times more inference tokens for MemFuse than naive RAG. This matters for attributing the gains to memory organization and assessing the performance-cost tradeoff.
Questions for authors
- Can you isolate the contribution of memory fusion and graph structure using a baseline with the models and budgets, but neither fused memories nor graph traversal? How does the comparison change under matched context-token and LLM-call/token budgets?
- Could you explain why wo/ fusion doesn't have much impact on the method?
- Please clarify the relationship to prior event-centric and episodic memory systems, including CompassMem [1], REMem [2], and Amory [3]. A more explicit comparison would help establish which contributions are specific to multi-source fusion.
- Figure 4 is not easy to read because the caption doesn't give much explanation for these abbreviations.
[1] Yuyang Hu, Jiongnan Liu, Jiejun Tan, Yutao Zhu, and Zhicheng Dou. “Memory Matters More: Event-Centric Memory as a Logic Map for Agent Searching and Reasoning.” Findings of ACL 2026 [2] Yiheng Shu, Saisri Padmaja Jonnalagedda, Xiang Gao, Bernal Jiménez Gutiérrez, Weijian Qi, Kamalika Das, Huan Sun, and Yu Su. “REMem: Reasoning with Episodic Memory in Language Agents.” ICLR 2026 [3] Yue Zhou, Xiaobo Guo, Belhassen Bayar, and Srinivasan H. Sengamedu. “Amory: Building Coherent Narrative-Driven Agent Memory through Agentic Reasoning.” EACL 2026
5 comments
Response to Q1: Separating the Structure from the Retrieval Controller
Thank you for your positive assessment and for suggesting this precise baseline.
In short. We ran exactly the baseline you described: under the same iterative controller, the structure adds less than about two points. We then isolated the structure by letting it perform the aggregation without any controller; on Gemini, removing it lowers Overall by 0.0485.
With the same iterative controller, hybrid retrieval, temporal/source constraints, and reader, adding fused memory and the graph changes Overall by less than about two points. This confirms your reading: once an LLM controller iteratively searches, it performs most of the cross-source aggregation itself.
This result motivated the revised retrieval (see General Response). Rather than letting a controller compensate for the structure, we use the structure as the aggregation mechanism and remove all retrieval-time LLM calls. In this setting the contribution of the structure is isolated: on Gemini 3.1 Flash Lite, with an identical retriever, reader, and 20-event budget, removing fused memory and the graph lowers Overall from 0.3636 to 0.3151 and gold-evidence recall@20 from 0.3565 to 0.2970. Graph propagation accounts for most of this (0.0306); fused grouping adds 0.0098 on top of the graph. Under GPT-4.1 Mini and Qwen3-30B-A3B, MemFuse exceeds naive RAG by 0.0183 and 0.0093 at the same reader budget, but under Qwen it still trails Mem0 and EverMemOS at k=20.
Summary. The structure's distinct value is clear once it, rather than an LLM controller, performs the aggregation. The size of the gain varies with the reader and ingestion model, and we report the Qwen result as a limitation.
Response to W2/Q1: Matched Computation and the Performance-Cost Trade-off
Thank you for this insightful point about computation.
In short. The revised retrieval removes this confound: MemFuse and naive RAG now use about the same inference tokens and no retrieval-time LLM calls, and MemFuse improves over naive RAG under all three readers. Compared at matched inference cost, MemFuse matches or exceeds the strongest memory baseline in every setting.
We agree with you that a shared item count did not equalize computation in the original configuration. In the revised retrieval:
- Matched reader context. MemFuse and naive RAG both pass at most 20 atomic events to the reader: 0.85 vs 0.84M, 0.77 vs 0.76M, and 0.90 vs 0.90M inference tokens over 357 questions (Gemini / GPT-4.1 Mini / Qwen3-30B-A3B).
- Matched retrieval computation. MemFuse makes no LLM calls at retrieval time; like naive RAG, it computes one query embedding, plus lightweight score aggregation over the stored graph. Each question uses one LLM call (the reader).
- Ingestion. All results reuse the memory snapshots of the submission, so ingestion cost is unchanged (29.73M–93.27M tokens, now reported in Table 2). This is a one-time offline cost that grows with the number of events, not the number of questions.
Under this matched budget, MemFuse improves over naive RAG by 0.0399 for Gemini, 0.0183 for GPT-4.1 Mini, and 0.0093 for Qwen3-30B-A3B. To address the performance-cost trade-off you raised, we also vary the reader budget (k = 20, 40, 80, 160, and 320 for all three readers) and compare with the other memory systems at matched inference cost (Figure 4 of the revision; tokens summed over 357 questions).
With larger budgets MemFuse also reaches the long-context reference, which reads the full event stream: 0.5373 vs 0.5223 for GPT-4.1 Mini at k=160 (4.91M tokens; long-context tokens are not available in this setting), 0.4733 vs 0.4424 for Qwen3-30B-A3B at k=160 (5.40M vs 40.83M tokens), and 0.5308 vs 0.5201 for Gemini at k=320 (10.50M vs 39.63M tokens). At k=320, MemFuse reaches 0.5606 under GPT-4.1 Mini (9.24M tokens) and 0.4774 under Qwen3-30B-A3B (10.32M tokens).
We do not attribute these budget gains to the fused structure: much of the improvement at larger k comes from passing more evidence to the reader, and the structure's measured contribution is the k=20 ablation reported above.
Summary. At matched reader tokens and retrieval computation, MemFuse improves over naive RAG under all three readers. Across budgets, it matches or exceeds the strongest memory baseline in each setting at lower or comparable inference cost. The revised Table 2 reports inference and ingestion tokens for every system, and its caption states the retrieval LLM calls.
Response to W1/Q2: Why Removing Fused Memory Has a Small Effect
Thank you for raising this; we agree it needs a clear explanation.
In short. In the submission, the agentic controller masked the effect of fused memory. In the revised retrieval the structure as a whole has a significant effect, and fused grouping and graph propagation overlap, so removing either one alone costs less than removing both.
As described in the General Response, we revised MemFuse's retrieval after the controller-matched comparison: the iterative LLM controller is replaced by LLM-free, query-time aggregation over the fused nodes and the graph, with the same reader budget as naive RAG. This change is what makes the effect of the structure measurable. In the submitted version, removing fused memory changed Overall by only 0.0080 because the iterative controller could recover the same evidence through additional search rounds. In the revised retrieval no controller can compensate, and the effect becomes visible: on Gemini 3.1 Flash Lite, removing fused memory and the graph together lowers Overall from 0.3636 to 0.3151.
The two components overlap. Both connect observations of the same episode: fused nodes group them at write time, and the graph links related activities at query time. Fused grouping therefore adds 0.0179 when the graph is absent but only 0.0098 when it is present. Before the reader, fused grouping has the larger effect: removing it lowers gold-evidence recall@20 from 0.3565 to 0.3219, more than removing the graph (0.3345). Part of this retrieval gain does not carry over to the answer score at k=20, because related records from different devices often repeat the same fact.
Summary. We present fused memory and the graph as one structure whose joint contribution is significant, rather than claim a large independent effect for fused grouping, and we revise the paper's framing accordingly.