acceptodds
Withdrawn from ICLR 2027

Membox: Weaving Topic Continuity into Long‑Range Memory for LLM Agents

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Remembering an interaction requires preserving how its parts belong together. Cognitive research describes how continuous experience is organized into structured events: event segmentation theory models the maintenance of an ongoing event representation and its updating at boundaries, connecting this organization to subsequent memory encoding zacks2007event. Discourse theory similarly explains coherent interpretation through the interaction of linguistic structure, communicative intentions, and attention grosz1986attention. In dialogue, a question, a reply, and a qualification may therefore belong to one unfolding exchange despite differences in their surface semantics. These perspectives motivate a construction question for agent memory: what should be preserved together as a retrievable unit? Local organization is only part of this problem. The event-indexing model of narrative comprehension describes connections among events along time, space, protagonists, causality, and intentionality zwaan1995construction. Drawing on these accounts, we formulate two complementary requirements for dialogue memory: preserve supporting exchanges within local units, and provide access between related experiences across unit boundaries. The resulting design must balance local coherence with cross-episode continuity as dialogue accumulates. Existing memory systems use enrichment, updates, indexing, and association to maintain useful representations and connections zhong2024memorybank,xu2025mem,chhikara2025mem0. These mechanisms can encode context, but retrieving a relevant record does not necessarily expose its surrounding exchange. Figure fig:existing illustrates the consequence: a question about continuing to run may require a training plan, recurring knee pain, and advice to reduce intensity. Retrieving the training and pain records can still omit the advice, whose importance comes from its role in the exchange rather than a strong direct match to the query. The facts may remain stored, while their joint availability depends on additional retrieval or linking.

Primary area:
applications to neuroscience & cognitive science
Keywords:
agent, memory

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_rYZKOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes Membox, a memory system for long-term dialogue. Topic Loom organizes original dialogue into boxes based on topic continuity and extracts features for retrieval. Trace Weaver links events across boxes to provide additional evidence at question-answering time. The paper evaluates the system primarily on LoCoMo, with cost and semantic accuracy analyses and supplementary experiments on DialSim.

Critical strengths
  1. Preserving local exchanges while connecting evidence across episodes addresses practical challenges posed by extraction omissions and fragmented evidence in long-term dialogue memory.
  2. Box achieves promising answer quality with lower memory-construction token usage than the locally reproduced Mem0/A-MEM baselines. The semantic accuracy gains from Trace provide further evidence of the system’s potential.
Critical weaknesses
  1. The results support the effectiveness of Box as a system, but I would appreciate clearer evidence of the additional benefit of adaptive topic segmentation over conventional retrieval of raw-text chunks. As the paper notes, topic segmentation and cross-memory association are established components. Comparisons with simpler alternatives and closely related approaches would help clarify the contribution of the proposed combination.
  2. I appreciate the retrieval-depth and expansion analyses and the semantic accuracy gains from Trace. However, the benefit of organizing events into traces beyond providing additional context remains unclear to me. For example, in Appendix Table 7, two GPT-4o-mini configurations using approximately 3,140 QA context tokens yield F1 scores of 53.95 for Box and 54.00 for Trace. Although this comparison does not cover all settings, comparisons with simpler ways of supplementing evidence would help clarify the added value of Trace.
Questions for authors
  1. How much additional benefit does adaptive topic segmentation provide over conventional retrieval methods that also retain the original dialogue? A comparison with standard chunking would help me better understand the specific contribution of Topic Loom.
  2. Building on the reported retrieval-depth and expansion analyses, could the authors compare Trace with simpler ways of adding evidence, such as directly retrieving relevant events, at comparable cost? This would strengthen my confidence in the value of organizing events into traces.
  3. The related work discusses methods for topic segmentation and cross-memory association, such as LightMem. Could the authors further clarify Membox’s key differences from these approaches and provide experimental comparisons? This would help me better understand the added value of the combined design.
Reviewer_yrpYOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes Membox, a long-term dialogue memory with two components. Topic Loom segments the dialogue online: for each new message, an LLM classifies it against the last two messages as continuous, partial shift or discontinuous, and a shift seals the current "box" of consecutive original messages, from which a topic, events and keywords are extracted for indexing. Trace Weaver links events across boxes into overlapping "traces" using embedding-similarity voting followed by LLM verification. At query time, boxes are retrieved by embedding similarity and their original text is given to the answering model (Box mode); Trace mode additionally adds all events of the traces reached through the top query-relevant events of the retrieved boxes. On LoCoMo (1,540 non-adversarial questions), Membox-Box with GPT-4o-mini reaches 53.95 F1 versus 40.95 for the best tested depth of a local Mem0 implementation, and Trace raises this to 55.28 with about 1,000 more context tokens; GPT-4o and DeepSeek-V3 results and an LLM-judge accuracy are also reported, with a supplementary DialSim evaluation.

Critical strengths

A simple, cheap design that returns original dialogue rather than extracted facts, with a clear motivating failure mode. The paper's observation that extracted-fact memories can lose context that only makes sense within an exchange (Figure 1) is well illustrated, and Box mode is a lightweight remedy. Construction costs about 236-255 LLM tokens per utterance, roughly an order of magnitude less than the local Mem0 and A-MEM implementations (Table 9), and the retrieval-depth and trace-expansion sweeps (Tables 6-7) are reported in full with context costs. This is consequential because it suggests that a simple original-text memory is a strong and inexpensive baseline that more complex systems should beat.

Critical weaknesses
  1. The experiments do not isolate the proposed contribution from simply retrieving original dialogue. Membox-Box returns raw conversation text, while the main baselines (Mem0, A-MEM) return extracted memory items, so the 13-point F1 gain may come from returning original text rather than from topic-continuous segmentation. The essential baselines are missing: retrieval over fixed-size windows of consecutive turns (e.g., 5-7 turns, matching the 4.9-6.6 utterances per box), retrieval over individual raw turns, and full-context answering. Without them, the paper cannot show that Topic Loom's LLM-based segmentation contributes anything. Comparisons are also limited to older systems: closely related topic- and event-organized memories that the paper itself cites (LightMem, StructMem, CAST, AtomMem, E-mem) are not compared, and the local Mem0 implementation scores well below the reported Mem0 row in Table 1 (e.g., temporal F1 34.52 vs. 48.93), which raises questions about baseline fidelity. This is consequential because the central claim is that topic-continuous organization is what helps, and the current evidence is consistent with a much simpler explanation.

  2. The Trace Weaver gain is small, not significance-tested, and disappears at matched context. With GPT-4o-mini, Trace at content top-k = 7 and event top-k = 2 reaches 54.00 F1 with 3,147 context tokens, essentially identical to Box at top-k = 10 (53.95 F1 with 3,131 tokens) (Table 7). With GPT-4o, the 1.7-point F1 gain and the 4.6-point judge gain come with nearly three times the context (2,390 -> 6,888 tokens), and no Box setting with matched context is reported. The GPT-4o-mini Box judge run is incomplete, so the semantic metric cannot assess the Box-to-Trace gain on that backbone, and all results are single runs without confidence intervals. This is consequential because Trace Weaver is half of the proposed method, and the evidence does not show that it improves over retrieving more boxes.

Questions for authors
  1. What are the results (F1 and judge accuracy) of retrieving fixed-size windows of consecutive original turns, with window size matched to the average box size and the same retrieval budget? And of full-context answering on LoCoMo?
  2. Can you report Box versus Trace at matched context budgets for GPT-4o (e.g., Box with top-k of 20-30), and confidence intervals or paired tests for the Box-to-Trace difference?
  3. How does Membox compare with LightMem or StructMem under the same LLM-judge protocol, given that the paper already uses LightMem's judging script?
  4. Why does the local Mem0 implementation score so much lower than the reported Mem0 results, particularly on temporal questions?
  5. How accurate is Topic Loom's segmentation against a dialogue segmentation benchmark (e.g., SuperDialseg), and how often do "partial shift" labels split a single exchange?
Reviewer_h8azOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper addresses a limitation of existing long-term dialogue memory systems: isolated storage and retrieval of records fail to represent connections across segments, leading to incomplete retrieval. To address this, the paper proposes a two-level memory construction design. Topic Loom segments session content into a set of initially disconnected boxes, while Trace Weaver uses events as cues to establish cross-box associations. At retrieval time, relevant boxes are first retrieved, and events within those boxes then index into traces to obtain additional cross-box information. The paper reports improvements over multiple baselines, supporting the effectiveness of the proposed memory mechanism.

Critical strengths
  1. Topic Loom organizes local session content into boxes, and Trace Weaver establishes connections across boxes via events. This design ensures that cross-segment connections are represented more explicitly, so retrieval is no longer confined to isolated records but can draw on broader and more complete evidence.

  2. The paper compares Membox against several baseline methods, and the results consistently demonstrate the benefits of the proposed memory mechanism. This provides reasonable evidence that the overall design is effective.

Critical weaknesses
  1. In line 185, when classifying a new message, the authors appear to consider only a window of the two most recent messages. I am concerned that this may make the classification result overly local and one-sided. For example, suppose Window 1 discusses a broad topic, Window 2 discusses Subtopic 1 under that broad topic, and the new message discusses Subtopic 2 under the same broad topic. If the similarity judgment finds that Subtopic 2 differs from Subtopic 1, the new message may be assigned to another box, even though it should belong to the same box under the broad topic in Window 1. This suggests that the box construction may be sensitive to the limited window and may split topic-continuous content incorrectly.

  2. My main concern is that the paper’s motivation is to address insufficient segment connections, and Trace Weaver is the most relevant design for this problem. However, in Table 1, Membox-box appears to contribute the largest portion of the improvement. Although Membox-trace yields further gains on top of Membox-box, the additional improvement is relatively small compared with the gain from the box component. As a result, the experiments seem to validate the advantage of Topic Loom over traditional baselines more than the effectiveness of Trace Weaver. Consequently, it remains unclear whether the segment-connection problem is actually well addressed by the proposed trace mechanism.

Questions for authors

See Weaknesses 1 & 2

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.