acceptodds
Under review as a conference paper at ICLR 2027

Growing Structured Memory with Learned Local Decisions for Online Ingestion

Abstract

In online memory construction, a corpus arrives incrementally and each new item must be ingested upon arrival. Explicit structure over items, such as knowledge graphs, can substantially improve retrieval, but growing and maintaining such structure under incremental arrival remains a challenge. Recent methods address this agentically, prompting an off-the-shelf LLM to perform each ingestion; this yields flexible updates, but the cost of these calls is high and grows with corpus size. We instead grow a structured memory through learned local decisions, coupling a hierarchy over the corpus, here a tree, with a model trained to maintain it and a retrieval procedure that exploits it. We cast ingestion as a local maintenance problem: each decision depends only on a bounded region of the tree, not on the memory as a whole. A 0.5B model, supervised-finetuned on decisions labeled offline by a larger model, is enough to make these decisions. Retrieval in turn draws on the hierarchy as a signal beyond flat similarity, recovering the benefit of structure without any LLM calls. We evaluate on standard benchmarks under both offline and online arrival, where it matches state-of-the-art memory systems in R@10 at a fraction of their ingestion cost. On CloneMem, RealMem and NovelQA it reaches R@10 of 35.70, 71.46 and 60.67, against 32.76, 67.86 and 57.56 for the strongest competing system, while spending less than a tenth of the hosted-LLM input tokens of the cheapest competitor and building its memory at least eight times faster.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Agent Memory, RAG, Deep Learning, LLMs, Memory System

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_xcLQOpenReview ↗
Rating
2
Confidence
2(−1)
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies online memory construction, where each new item must be ingested on arrival. Existing agentic memory systems call an LLM for every ingestion decision, so their cost grows with the corpus. Thus, the authors propose to grow a tree-structured memory through two learned local decisions per item, (1) routing the item from the root to a position in the tree, and (2) reorganizing a small window around it with IDLE, PROMOTE, DEMOTE or GROUP edits. Both decisions are made by a 0.5B policy distilled from a 27B teacher. Retrieval combines dense and lexical search, hand-designed tree factors and a reranker. On CloneMem, RealMem and NovelQA, the method outperformsHippoRAG 2 while using less than a tenth of the hosted-LLM tokens of the cheapest competitor.

Critical strengths
  1. Turning online ingestion into small local decisions and distilling them into a 0.5B model is a simple and practical idea. It removes most per-item LLM calls, and the savings are large. The method uses over 10x fewer hosted-LLM tokens and builds its memory at least 8x faster than the agentic memory baselines. Table 6 also shows that the local window stays small as the tree grows, which supports the locality claim.

  2. The distillation works well. On held-out decisions the 0.5B policy reaches 78.8% accuracy, close to the 27B teacher's 80.6%. Random legal actions (21.7%) and the untrained base model are far behind. This suggests a small model is enough for these local decisions.

  3. The design is clean. Grammar-constrained decoding makes every edit legal by construction, so a bad output can't break the tree. The weighted loss focuses on the few tokens that actually carry the decision, which is a sensible fix for answers that are mostly template.

  4. The paper is well written and easy to reproduce. Figures 1 and 2 explain the method clearly, and the appendix gives all the prompts, training settings and data sources.

Critical weaknesses
  1. I am concerned that most of the gain may not come from the tree. By the paper's own numbers, BM25 and the reranker add about 7 to 16 points of R@10. The tree factors add less than 1 point (+0.92, +0.25, +0.18), and that gain is only significant on CloneMem. A flat retriever with the same BM25 and reranker would likely already beat all the baselines. So it is hard to credit the results to the structured memory, which is the main claim.

  2. Meanwhile, the baselines don't use the same retrieval stack. The proposed method uses Qwen3-Reranker-4B, but none of the memory baselines gets a comparable reranker. As a result, Table 1 compares retrieval pipelines as much as it compares memory organizations.

  3. The value of the learned policy isn't shown directly. The policy is only checked by its agreement with the teacher on 500 decisions. The paper never compares it with trees built in simpler ways, such as MemTree, RAPTOR or even random grouping, under the same retrieval. MemTree is described as the closest work but isn't used as a baseline.

  4. The cost comparison is unfair and leaves out important parts. Teacher labeling took about 6,500 B200 GPU-hours and training used up to 40 B200 GPUs, but neither shows up in the main cost claims. The 4B reranker at query time isn't counted either, even though the abstract says retrieval needs no LLM calls.

  5. The evaluation only uses R@10. There's no metrics for QA accuracy, etc. Mem0 and A-Mem are built to return compressed memories for answering questions, so judging them only by recall over mapped-back sources may not be fully fair to them, I think?

Questions for authors

I would be thankful and willing to raise my score if the authors can (partly) address the following questions/concerns.

  1. Could add baselines like "Flat + BM25 + reranker" with no tree, and ideally give HippoRAG 2 the same reranker? If the tree still helps clearly on at least two benchmarks, my concern would be resolved.

  2. Under the same retrieval, how does the learned tree compare with trees built by (1) the teacher, (2) by MemTree-style similarity insertion, or (3) by random grouping?

  3. Most of the tree's gain is on latest-state questions and comes through F3. Would a simple recency boost without any tree give similar gains?

  4. How well does the policy perform on the benchmark trees themselves, which differ from the training data? Does tree quality drop as more items arrive?

  5. Could the authors report the full cost, including teacher labeling, training and the reranker? An estimate of how many ingested items it takes for the method to pay off would be better. Meanwhile, why is retrieval on RealMem so slow?

  6. A small confusion I may have made. The gains in Sec. 4.2 are measured from Flat, but Table 2 uses Base (cosine), and the two start from slightly different numbers. Do they retrieve over different units or the same?

2 comments
AuthorsOpenReview ↗

Response to Reviewer xcLQ (1/2)

We thank the reviewer for the careful review.

W1 & Q1: Is the gain due to the tree, rather than BM25 and the reranker?

Please see General Response A.

W2: The baselines do not use the same retrieval stack.

Now we are all fair. Please see General Response C.

W3 & Q2: How does the learned tree compare with simpler trees under the same retrieval: trees built by the teacher, by MemTree-style insertion, by random grouping, and by RAPTOR?

Table R1 in the General Response (B) compares the learned tree with these simpler trees under the same retrieval pipeline. Beyond the 500 judged decisions, we also measure agreement with the teacher on 18,104 held-out reorganization decisions (App. C).

  • (1) Teacher-built trees. Building them at benchmark scale is too slow: with thinking, the 27B teacher needs about 1–1.5 days on one GPU for the online benchmarks. At the decision level, the policy reaches 97.8% of the teacher's accuracy on held-out decisions (78.8% vs. 80.6%).
  • (2) MemTree insertion (General Response, B).
  • RAPTOR. We did not run RAPTOR. It builds its tree offline by recursively clustering and summarizing the whole corpus, so it is not suited to online memory updates: each new item would require rebuilding the tree.

Q3: Would a simple recency boost without any tree give similar gains, given that much of the gain appears on latest-state questions through F3?

  • Latest-state questions. We add the exponential recency score of Generative Agents (Park et al., 2023), γaγ^{a} with aa the number of items that arrived later, weighted as F3, with γ and the weight selected in the same way as F3's parameters (γ = 0.97). On latest-state questions (C3), it reaches 53.51 pooled R@10 against 54.30 for F3 (Table R8). The difference (+0.79) is not significant.
  • Full pipeline. With the reranker, replacing all three tree factors by this recency score gives 35.15 / 71.57 / 60.52 R@10, close to ours (35.52 ± 0.28 / 71.31 ± 0.13 / 60.59 ± 0.20). We therefore do not claim that F3's gain needs the tree: it is mostly a recency effect. Designing factors that use the learned structure beyond arrival order is future work.

Table R8. On latest-state questions (C3), R@10 of F3 on the learned tree, of an exponential recency boost (Park et al., 2023), and of F3 without the tree (it lifts the newest items of the whole stream); cosine base, no BM25 or reranker.

CloneMemRealMemPooled
F3 on the learned tree42.8665.7354.30
Exponential recency (Park et al., 2023)41.5265.5053.51
F3 without the tree (newest items of the stream)39.7664.2952.02

Q4: How well does the policy perform on the benchmark trees, which differ from the training data, and does tree quality drop as more items arrive?

The benchmark trees have no teacher labels, so we measure their quality by what the tree adds to retrieval. Table R9 splits the gain of the tree factors by the number of items N in the tree when each question was asked, on the two online benchmarks (NovelQA questions are asked only after a whole book has been inserted). It is a reasonable concern, in experiment we observed that the descent was not obvious.

Table R9. R@10 of Flat + BM25 + reranker (no tree) and of Ours, by the number of items N in the tree when the question was asked (averaged over the queries in each bucket). Ours: mean ± standard deviation over ten end-to-end runs.

BenchmarkNQueriesFlat + BM25 + reranker (no tree)Ours
CloneMem<1005370.5571.22 ± 0.50
CloneMem100–30032037.6538.68 ± 0.25
CloneMem300–1k66029.8230.60 ± 0.34
CloneMem1k–3k15421.2421.28 ± 0.33
RealMem<10068281.2381.35 ± 0.15
RealMem100–30073361.3961.50 ± 0.26

Dear Authors,

Thank you for the comprehensive response and the extensive new experiments. I appreciate that you unified the retrieval stack across all baselines, provided the "Flat + BM25 + reranker" comparison, and clarified the training and labeling costs.

While these additions help address my primary concerns regarding the fair evaluation of the tree structure and the actual system overhead, after carefully considering the new data and the nuances of the results, I have decided to maintain my current score. However, given the additional complexities and technical details discussed in the rebuttal, I will be lowering my confidence score.

Regards,

Reviewer xcLQ

Reviewer_iqrTOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies how to build and maintain a searchable memory tree as records arrive. A 27B teacher supplies decision labels for a 0.5B policy that routes each new record through the tree and reorganizes nearby records by keeping, promoting, demoting, or grouping them. Constrained decoding enforces legal actions, while gpt-4o-mini still names newly created categories. Retrieval combines embedding similarity, BM25, three fixed tree-based scoring factors, and a Qwen3 reranker, returning the original records. The authors evaluate evidence R@10 online on CloneMem and RealMem and offline on NovelQA. The full system matches or exceeds the four comparison systems in retrieval quality while substantially reducing deployment-time memory construction costs. Ablations show that the tree adds only small gains on top of the stronger retrieval pipeline.

Critical strengths
  1. The deployment cost–quality trade-off is promising. The full system matches or exceeds the four comparison systems in R@10 on all three benchmarks and builds memory at least eight times faster on the timed subsets. This is useful for continuous ingestion where frequent large-model calls are expensive.

  2. There is initial evidence that a small model can handle structural maintenance. One 0.5B policy performs both routing and reorganization, with constrained decoding enforcing legal actions. In the LLM-judged evaluation of held-out decisions, its accuracy is 78.8%, close to the teacher's 80.6%, supporting the feasibility of distilling these local decisions.

Critical weaknesses
  1. The added value of learned tree maintenance is not established. Over the same cosine + BM25 + reranker pipeline, tree factors add only 0.92/0.25/0.18 R@10 percentage points. Without corresponding comparisons to a rule-based online tree or a version without reorganization, it remains unclear whether these gains require the trained maintenance policy.

  2. The scalability claims are stronger than the evidence. Appendix E explicitly acknowledges that window size is unbounded, and Table 6 reports a maximum of 1,484 nodes. Build timings on three buckets per benchmark do not establish per-item tail latency as memory grows, leaving bounded local work and long-term scalability unverified.

Questions for authors
  1. Under the same retrieval pipeline, how does a rule-based online tree, such as routing inspired by MemTree, or a version without Stage 2 reorganization perform? Could you also report build and query costs for these variants, the current system, and the existing tree-free hybrid + reranker control?

  2. How do median/p95 ingestion latency and policy-input token length change as memory grows? What happens when a local window exceeds the 32,768-token context, and does this handling affect any reported results?

  3. Could you report the paired differences and confidence intervals against cosine + BM25 + reranker directly for all three test benchmarks, and specify the bootstrap resampling unit? For the persona-averaged online benchmarks, how is correlation among queries from the same user and tree handled?

2 comments
AuthorsOpenReview ↗

Response to Reviewer iqrT

We thank the reviewer for the careful review. W1 and Q1 overlap with the shared concerns in the General Response, so we answer them briefly here and point to the details there.

W1 & Q1: Is the trained policy needed? How do a rule-based tree and a version without Stage 2 perform, and what do all variants cost?

  • Rule-based tree and the version without Stage 2 (General Response, B; Table R1).
  • Why the gain is small with the reranker (General Response, A and B). The reranker re-scores the same top 40 candidates, so different trees score close to each other.
  • Build and query costs (General Response, D; Table R2). Table R2 reports the build time, retrieval time and LLM API tokens.

W2 & Q2: Do the scalability claims go beyond the evidence (an unbounded window, a 1,484-node window in Table 6, no per-item tail latency as memory grows)? How do latency and input length change as N grows, and what happens to windows beyond 32,768 tokens?

  • Bounded local work. Thanks for the reviewer’s findings. We found by adding a width cap: when a node has more than M = a constant number of children, they are clustered by embedding into new categories, named as for GROUP. After that, a closure then has at most 2M+2 nodes, which made it to be bounded. We will add this into revision.
  • Input length vs. N. Table R5 gives the policy input by N. The Stage 1 input stays almost constant as N grows (median 594 → 627 tokens on CloneMem, 404 → 453 on NovelQA), and the Stage 2 window grows only modestly (median 2,087 → 2,380 on NovelQA). The 1,484-node window of Table 6 is one category in the largest book (B48, 14,158 items) that holds 986 items. Today, a window too large for the context is sent as several batched requests. With the width cap, no window holds more than M items, and batching remains only for windows of long items, as on RealMem.

Table R5. Policy input against the number of items N already in the tree, rebuilt from all 1,235 snapshots (sampled as in Table 6; token counts match vLLM's on all 126 requests we could check).

BenchmarkNS1 tokens/request, medianS2 window tokens, median / p95Max items under one parent
CloneMem<1005943,242 / 8,95831
CloneMem1k–3k6275,696 / 30,495106
RealMem<1001,26416,314 / 71,29161
RealMem100–3001,34410,830 / 84,54771
NovelQA300–1k4042,087 / 11,677232
NovelQA≥3k4532,380 / 23,764986
  • Windows beyond 32,768 tokens. None of our 364,831 requests exceeded the context. Stage 1 is never split; its longest input is 24,849 tokens. We’ve already considered it, so we use a batch-request to deal with windows beyond 32768 to avoid this issue. Moreover, by adding the M-constrain, it will no longer be a problem.
  • Effect on reported results. No reported comparison changes. On B48 and B31, the full system and the tree-free control (Dense + BM25 + reranker (no tree)) have similar R@10, and dropping both books moves the NovelQA tree gain from +0.10 to +0.11. App. E will describe the splitting, the output cap and the width cap.
  • Latency vs. N. In the serial RTX 4090 run, the per-item latency does not grow with the tree: p50 0.35 / 0.36 / 0.30 s and p95 2.08 / 1.69 / 1.39 s for N < 100, 100–300 and 300–435. In the largest tree, the policy keeps 4.1–4.7 requests per item, and apart from the Stage 2 answers that hit the output cap above, which the width cap removes, the time per item stays at 0.5–0.8 s.

Q3: Paired differences and CIs against cosine + BM25 + reranker, the bootstrap resampling unit, and correlation within a user and a tree.

Table R7. R@10 of each pipeline and the paired gain Δ from the tree factors (percentage points). ± is the standard deviation over ten end-to-end runs (General Response, A); the pipeline without the tree is deterministic.

CloneMemRealMemNovelQA
Cosine + BM25 + reranker (no tree)34.7871.2160.49
Ours (cosine + BM25 + tree factors + reranker)35.52 ± 0.2871.31 ± 0.1360.59 ± 0.20
Δ with reranker (row 5 − row 4)+0.74 ± 0.28+0.10 ± 0.13+0.10 ± 0.20
Queries1,1871,4151,541
  • Bootstrap:We Rerun in three datasets, please see the General Response A.

We hope these responses address your concerns, and we would be glad to clarify anything further.

Thank you for your response. However, MemTree matches your learned policy with lower latency and token costs. Benefit of learned maintenance is unclear. I maintain my current score.

Reviewer_cTNNOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

This paper proposes a tree-structured memory organization for online memory building (where facts arrive in a streaming fashion), as a more efficient version of graph-based methods. When a new memory arrives, it reduces LLM inference calls by only considering one root-to-leaf path in the tree, eliminating much of the existing memory from consideration. It further reduces cost by training a small 0.5B model for making these local tree-exploration decisions.

Experiments on 3 datasets (CloneMem, RealMem, NovelQA) show superior performance compared to other memory organization approaches such as Mem0, A-Mem, MemOS, and HippoRAG 2. Compared to the flat baseline, however, most of the gains came from adding a hybrid retriever (dense+BM25) and a reranker (+9.95 pts on average), whereas the tree structure only adds 0.45 pts. (The efficiency of the dense+bm25+reranking flat baseline is not shown.)

Critical strengths
  • The proposed tree structure that enables memory building without traversing the entire graph is interesting, and shows promising results compared to other graph-based methods using only a 0.5B model. The token saving over graph baselines is significant while achieving superior accuracy.
Critical weaknesses
  • In the evaluated system, the tree plays a small role. Line 421 states that the tree structure only adds an average of 0.45 points on 3 datasets compared to a flat baseline that uses the same retrieval as the proposed method (dense+bm25+reranker). That flat baseline achieves significantly better accuracy compared to all graph baselines, uses 0 LLM tokens, and presumably faster than the proposed method. The paper does not convincingly establish that added cost and complexity is well justified over a competitive flat baseline.

  • The proposed tree structure may not be as expressive as more general graph-based methods, such as HippoRAG 2. For example, a single memory / fact may span multiple axes. Suppose the memory contains "hiking with my partner", "hiking with my colleagues", "painting with my partner". If the tree is organized by activity, it works well for query "which hikes did I go" but struggles with "what did I do with my partner". Organizing by people reverses the problem. In practice, this issue is mitigated in this paper, since this paper is not doing retrieval mainly based on the tree structure (See weakness 1). Instead, it is a standard flat retrieval (cosine), with an intermediate reranking stage using the tree structure, followed by another standard (flat) reranker. Table 2 shows that, adding the tree structure (F1+F2+F3) gives only +1.4 points over the flat baseline, and the majority of the gain is again from BM25+reranker. In a more tree-dependent system (e.g. retrieval dependent on the tree structure), the relevant trade-offs remain untested. Overlap and associative retrieval could become problematic, especially with hard subtree pruning.

Questions for authors

Q1: What's the build/retrieve time for the dense+bm25+reranker flat baseline?

2 comments
AuthorsOpenReview ↗

Response to Reviewer cTNN

We thank the reviewer for the careful review.

W1: The tree adds little over the flat baseline (Dense + BM25 + reranker (no tree)), the flat baseline beats the graph baselines, and the added cost may not be justified.

  • The gain is small but not random (General Response, A). With the reranker, it is concentrated on multi-evidence questions on CloneMem (C2: +0.68), the case that F2 is designed for.
  • Why the flat baseline beats the other memory systems (General Response, C). Our reproductions of Mem0 and A-Mem are close to the published numbers (Table R4 in the General Response). These systems compress information during memory construction, so details can be lost. HippoRAG 2 keeps the original passages, and with the same reranker it is not significantly below the flat baseline.
  • Cost (General Response, D, including Table R2). Both pipelines run the same local 4B reranker and retrieve in about the same time. In our added experiments, every system uses the same local reranker, so we do not count its cost for any of them. What remains is the LLM API cost of memory construction. For us, this is only gpt-4o-mini labeling new category nodes when items are inserted: 9.9M input tokens over all three benchmarks, less than $2. The gain is small, but it comes from a clear mechanism: the tree uses information that text matching cannot see (General Response, A). For this gain, the extra cost is a 0.5B policy on one RTX 4090 during memory construction, and only CPU computation during retrieval.

W2: A single-parent tree is less expressive than a graph, and the trade-off is untested when retrieval depends on the tree.

We see this as a trade-off rather than a weakness of the tree. A graph supports more relations. But tree operations are much simpler. During memory construction, a 0.5B policy learns them easily. During retrieval, it is clear for us to make use of the tree structure to develop some tree factors.

We agree that the disadvantage of tree structure, including one memory item only linked with one category, cannot be overcome by tree factors. Which means we cannot solve the multi-evidence problem. However, what the structure tree provides can let us use factors to solve some other problems. For example, on CloneMem a lookup question asks "When you were looking through the job boards that night, were there any companies that really stood out to you?". The floor matches the user's diary entry from that evening about tweaking a resume, but the answer is the user's search history from the same evening (operations-director requirements and job openings), which shares few words with the question; the flat pipeline ranks it 53rd. The policy filed both records into the same leaf category ("Career Transition Dilemmas"), two arrivals apart, so F1 lifts the search history to the top (first in the original run, and in the top 10 in 8 of the 10 runs).

Q1: Build and retrieval time of the flat baseline.

Table R2 in the General Response (B) reports both. The flat baseline builds faster than ours, since no model runs. Both retrieve in about the same time, since the same reranker dominates.

We hope these responses address your concerns, and we would be glad to clarify anything further.

Thank you for your response. I appreciate the honesty in acknowledging the small gains brought by the tree structure. While the authors show that the gain is not random, the scale of the gain is minimal. I maintain my weak rejection score because i) the sub-1-point gain is hard to justify the additional complexity and 2x-5x slowdown at inference time, and ii) the paper would likely need a substantial rewriting to remove the misleading positioning, and honestly present the real contributions given the much smaller accuracy advantage.

Comments on the paper

2 comments
AuthorsOpenReview ↗

General Response (1/2)

We thank the reviewers for their constructive feedback. We appreciate that they find the token savings over the baselines significant (cTNN), the deployment cost–quality trade-off promising (iqrT), and the idea of casting ingestion as small local decisions simple and practical (xcLQ). We first address the concerns common to several reviews.

A. Contribution of the tree beyond a flat retrieval pipeline (cTNN W1; iqrT W1; xcLQ W1, Q1, Q3)

We agree that the tree factors add only a small average gain over the flat dense + BM25 + reranker pipeline. However, unlike this pipeline, which relies only on query–item relevance, the tree factors use the structure that the ingestion policy builds and locally revises as items arrive, namely category membership and arrival order. This information remains useful on top of the reranker: on CloneMem the tree factors add +0.74. Answering such questions requires retrieving several pieces of evidence, yet the flat pipeline scores each candidate independently of the others. F2 (event context) instead promotes candidates that share a category with, or arrived close in time to, a top-ranked candidate, even when they are only weakly similar to the query.

To measure variance, we repeated the whole pipeline end to end on each dataset: the ingestion policy rebuilt every tree from scratch, and retrieval were re-run on each, so the runs differ only through LLM inference. Over ten such runs, our R@10 is 35.52 ± 0.28 on CloneMem, 71.31 ± 0.13 on RealMem and 60.59 ± 0.20 on NovelQA (mean ± standard deviation over runs).

In summary, the value of the tree lies in structure that the policy maintains online through local decisions and that retrieval then exploits beyond query–item relevance. The design is also extensible: ingestion decisions could be tailored to these factors, or the factors could be learned from data, which we leave for future work.

B. Comparison of learned tree maintenance with rule-based tree construction (iqrT W1, Q1; xcLQ W3, Q2)

We compare the tree maintained by our learned policy with two alternative trees, keeping the rest of the retrieval pipeline fixed (Table R1). No tree: our system without Stage 2, where every item stays under the root, so it reduces to dense + BM25 + reranker. MemTree follows its original insertion rule; we only tune its base threshold θ₀ on each benchmark to match the number of categories in our tree, for a fair comparison.

Table R1. R@10 with the same retrieval pipeline and reranker; only the way the tree is constructed differs. Ours: mean ± standard deviation over ten end-to-end runs (General Response, A); the other rows are single runs.

Tree ConstructionCloneMemRealMemNovelQA
No Tree34.7871.2160.49
MemTree35.5871.2760.59
Ours35.52 ± 0.2871.31 ± 0.1360.59 ± 0.20

Table R2. Build / retrieve time (s) and hosted-LLM input tokens of the three pipelines under the same retrieval stack and reranker, following the same protocol as the main experiments (Table 1 in main paper).

PipelineCloneMemRealMemNovelQAHosted-LLM tokens (in)
No Tree15.0 s / 91.9 s21.8 s / 355.0 s13.3 s / 32.9 s0 / 0 / 0
MemTree220.3 s / 82.3 s148.2 s / 324.1 s111.4 s / 30.0 s0.85M / 1.04M / 1.64M
Learned policy (ours)558 s / 258 s554 s / 1537 s538 s / 64 s1.40M / 1.10M / 7.41M
AuthorsOpenReview ↗

General Response (2/2)

C. Fair retrieval with other memory systems (cTNN W1, Q1; iqrT Q1; xcLQ W2, W4, Q5)

  • Other memory systems. CloneMem and RealMem are existing benchmarks on which their authors evaluate Mem0 and A-Mem with the same evidence recall. Our adapters map every returned memory back to the source items it was built from, and our reproductions of Mem0 and A-Mem are within 1.2 points of, or above, the published numbers (Table R4). On CloneMem and RealMem, Mem0, A-Mem and MemOS fall below Flat, which keeps the original items; this is consistent with, though it does not prove, a loss of evidence when memories are compressed at ingestion.
  • Matched retrieval. With the same reranker, ours is on par with HippoRAG 2 on the two online benchmarks (+0.05 / +0.60), while using under 1/10 of HippoRAG 2's hosted-LLM tokens and building over 8× faster (Table R3).

Table R3. R@10 of all systems with the same Qwen3-Reranker-4B. Parentheses show the gain of the tree factors over Dense + BM25 + reranker (no tree). Ours: mean ± standard deviation over ten end-to-end runs (General Response, A).

SystemCloneMemRealMemNovelQA
Mem034.1764.6752.66
A-Mem33.8461.7342.86
MemOS31.4962.5360.62
HippoRAG 235.4770.7164.40
Dense + BM25 + reranker (no tree)34.7871.2160.49
Dense + BM25 + reranker + tree factors (Ours)35.52 ± 0.28 (+0.74)71.31 ± 0.13 (+0.10)60.59 ± 0.20 (+0.10)

Table R4. Published numbers versus our reproductions (R@10) for the same method and benchmark.

PublishedOurs
CloneMem, Flat27.5227.96
CloneMem, A-Mem26.8725.71
CloneMem, Mem017.5424.61
RealMem, Mem060.5560.74
RealMem, A-Mem57.6060.43

D. Cost

  • All pipelines retrieve in about the same time, since the same reranker dominates (Table R2). Both pipelines run the same local 4B reranker, and our only additional LLM use is naming new categories (9.9M gpt-4o-mini input tokens over all three benchmarks, under $2 at list price). Labeling and training need far less than we spent: with 10% of the labels, the policy reaches 76.4% attach accuracy against 81.6% with all labels.
  • We spent about 6,500 B200 GPU-hours on labeling and 294 on training, but most of this is not needed. With 10% of the labels and the same training process, the policy reaches 76.4% attach accuracy and 75.6% reorganization accuracy, against 81.6% and 77.2% with all labels. We estimate that labeling and training then take about two weeks on 8 A100 GPUs.
  • For reranker, It is a 4B model run locally, with up to 40 calls per query. Its total input is 27.7M / 82.1M / 19.6M tokens on the three benchmarks, nearly the same as for Flat + BM25 + reranker (no tree). The comparison with other memory systems does not rest on it. We will state in the abstract that retrieval runs a local 4B reranker used some tokens.

E. Using RAG task is still fair with other memory systems

On these benchmarks, finding the evidence is largely what answering requires. A question whose evidence is retrieved can be answered, and a question whose evidence is missing generally cannot. For example, NovelQA asks where George was when Alice and Kate got upon the bridge, and only the passage saying that he "strolled behind them" answers it. RealMem asks when the user can fit in the writing sprints discussed earlier; the answer depends on an earlier session in which the user said that a soccer tournament takes the whole weekend. R@10 therefore measures what QA accuracy depends on.

By this problem choosing and setting, it is fair for other memory system(such as Mem0,A-mem,etc)

We thank the reviewers again, and we would be glad to discuss any further questions.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.