acceptodds
Under review as a conference paper at ICLR 2027

MemSeeker: Query-Time Memory Construction

Abstract

Memory for LLM agents often relies on prebuilt semantic stores or query-time search over raw histories. Prebuilt stores incur upfront construction costs before query needs are known, while stateless search agents repeatedly spend effort locating and synthesizing evidence that earlier queries have already uncovered. To address these limitations, we define Query-Time Memory Construction (QTMC) as a memory construction strategy that starts with an empty store and incrementally constructs persistent memory from completed query trajectories for use by subsequent queries. We introduce MemSeeker, an implementation of QTMC that unifies efficient agentic search and persistent memory construction through structured search trajectories. Its cited working notes support efficient reasoning during search and serve as inputs to deterministic memory construction after answering. Evaluated on 2,400 questions spanning eight heterogeneous benchmarks, MemSeeker uses 83.5% fewer tokens while retaining comparable accuracy to the strongest baseline. Further analysis identifies two ways persistent memory supports subsequent queries: supplying evidence for direct answers and guiding the search for missing information. Code is available at https://anonymous.4open.science/r/Memseeker-Query-Time-Memory-Construction-0D3B.

Primary area:
foundation or frontier models, including LLMs
Keywords:
large language model agents, long-term memory, test-time memory construction, agentic retrieval, cross-query evidence reuse

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_ZR32OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies when an agent should construct persistent memory. Preparing a memory store before any questions arrive can spend effort on unused information, while starting a fresh search for every question discards useful work. MemSeeker keeps cited working notes during search and turns them into persistent memory after answering, without another LLM extraction pass. Later questions can use these memories directly or use them to guide further search.

Critical strengths

I find the construction timing worth studying. The current question tells the agent what information is useful, and the search has already paid the cost of finding and organizing it. MemSeeker uses this work twice, instead of running a separate memory-building process afterward.

Critical weaknesses

The benefit of persistence on general query sequences is not yet well separated from better single-query search.
The full system changes both the organization of the search state and whether knowledge is saved across questions. The dedicated MuSiQue experiment creates explicit reuse opportunities: later questions have earlier anchor questions with equal or nested supporting evidence, and the questions are further filtered. This is a reasonable way to test whether reuse can work, but it does not tell us how much it usually helps without that selection.

The general benchmark suite does not report the same MemSeeker agent with only cross-query persistence turned off. We therefore cannot attribute the full 83.5% token saving to persistent memory; some of it can come from more efficient search within each question.

Questions for authors

On existing corpora with multiple queries, what happens if the search agent is kept unchanged and only cross-query persistence is disabled? Breaking down the result by evidence overlap between questions would help show when saving the working notes is useful.

3 comments
AuthorsOpenReview ↗

Isolating the contribution of cross-query persistence

Thank you for suggesting this comparison. We added MemSeeker w/o HET result on the general suite, keeping the within-query search agent unchanged and disabling cross-query persistence. On the general suite, HET reduces tokens per query from 46.89k to 44.03k (6.1%), with comparable scores (70.38 vs. 70.52, Table 1). This isolates persistence’s incremental benefit; the 83.5% saving over DCI-lite measures the full system’s efficiency.

The 6.1% aggregate averages questions that can reuse prior evidence with questions that cannot; grouping them by overlap separates that average. Table 2 examines 415 questions from multi-query corpora that require agentic search without HET, selected independently of HET-side behavior. The analysis retains 134 zero-overlap cases, which incur a small 2.1% overhead. Nonzero-overlap groups save 12.0%–50.2%, with larger savings generally observed at higher overlap. Notably, 82 questions requiring search without HET are answered directly after initial retrieval with HET, saving 87.1% of tokens.

Reuse opportunities are limited in these benchmark protocols. CL, CL-Life, and LongMemEval provide one query per corpus in our evaluation. Appendix D discusses how other benchmarks tie questions to specific content or evidence, with some construction procedures explicitly limiting redundancy.

The new case studies in Appendix E.2 show how cross-query connections help without deliberately constructing overlapping question pairs. Retained purchase records enable a correct direct answer in Memora (74.3k → 22.7k tokens); a retained event timeline guides retrieval of missing actions in AMA (nine → one additional search round, with a correct answer). These illustrate two benefits of persistence: answering from previous findings and focusing subsequent search.

These observations are consistent with the MuSiQue results. General-suite overlap is measured from agent trajectories; MuSiQue’s gold supporting-evidence annotations allow us to control reuse opportunities and examine these mechanisms more precisely. Together, the experiments distinguish the overall benefit on existing query streams from the gains achievable when successive questions share reusable evidence.

Thank you for running the same search agent with persistence disabled and breaking down the results by evidence overlap. I’m keeping my rating at 2 because enabling persistence saves 6.1% of tokens on the general suite at essentially unchanged accuracy, while the larger gains on MuSiQue come from questions deliberately paired to share evidence.

AuthorsOpenReview ↗

Interpretation of 6.1% of token saving, MuSiQue reuse design, and the contribution of QTMC

Thank you for the follow-up. We would like to clarify three points about the interpretation of these findings.

  1. Unchanged accuracy and 6.1% aggregate savings. HET is designed to avoid repeated search and synthesis, so maintaining accuracy at lower token cost demonstrates its intended benefit. The aggregate saving is indeed 6.1%, but it averages queries with and without opportunities for reuse. Our general-suite breakdown already separates these cases and establishes where persistence helps (Table 2).
  2. What the MuSiQue experiment establishes. MuSiQue deliberately controls reuse opportunities, but paragraph-level evidence overlap does not imply identical information needs or automatic answer reuse. Questions sharing supporting paragraphs may require different facts; others require selecting intermediate conclusions or extending them with new evidence. Successful reuse therefore depends on what knowledge HET retains and how the agent applies it. This experiment isolates those mechanisms, while the general suite examines their benefits under heterogeneous workloads.
  3. QTMC’s central contribution is constructing memory from the work performed to answer actual queries. In real settings, memory access to one corpus (like one area in the raw history) can be as sparse as LME, CL, etc, or as dense as designed MuSiQue. MaS always incurs construction costs before knowing what will be asked, processing the whole corpus even if only one query will refer to it in the future. Stateless MaA operate when query arrives, but the search cost grows linearly as queries arrive. MemSeeker uses the same structured working notes for efficient search and subsequent memory construction. Single-query corpora benefit from avoiding upfront construction; later queries with reusable evidence can additionally avoid repeated work. Evaluating QTMC solely by the 6.1% persistence increment leaves out this central contribution: making memory construction demand-driven and integrating it with efficient query serving.

If you are satisfied with our analysis, we would greatly appreciate your re-consideration in adjusting the evaluation scores accordingly. We sincerely look forward to your feedback.

Reviewer_voDnOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper focuses on agentic memory. The authors argue that prebuilt memory stores can incur substantial construction costs before query needs are known, while stateless search agents discard intermediate findings and repeat retrieval and synthesis for later queries with overlapping information needs. They propose Query-Time Memory Construction (QTMC) and introduce MemSeeker, which starts with an empty memory store and incrementally constructs persistent memory from completed query trajectories. MemSeeker deterministically converts cited working notes into a Hierarchical Evidence Tree (HET), preserving source citations and claim revision history without an additional LLM extraction pass. Experiments across eight benchmarks show substantially lower token usage while maintaining average performance comparable to the strongest evaluated baseline.

Critical strengths
  • An efficient integration of search and memory construction. MemSeeker reuses cited working notes generated during search to construct persistent memory deterministically. This avoids additional LLM calls for memory extraction while preserving source citations and claim revision history.
  • Broad benchmark coverage supported by informative ablations. The evaluation covers eight heterogeneous benchmarks, while controlled ablations separately examine query-local context management and cross-query memory reuse.
Critical weaknesses
  1. QTMC is insufficiently distinguished from prior memory methods, and a key experimental comparison is missing. Accumulating persistent memory from completed tasks and reusing it for subsequent queries has been explored by G-Memory (NeurIPS 2025) and ReasoningBank (ICLR 2026). Although MemSeeker introduces a deterministic protocol for materializing cited working notes without additional LLM calls, the paper neither discusses these works nor establishes its incremental contribution relative to them. The experiments also include HippoRAG, citing its original version, but omit HippoRAG 2 (ICML 2025), a relevant successor for the document QA workloads. These gaps limit the assessment of both QTMC’s conceptual contribution and MemSeeker’s comparative advantages.
  2. Cross-query consistency and error correction remain insufficiently addressed. HET links claim revisions only within individual query trajectories, leaving claims from different queries independently retrievable without explicit reconciliation or invalidation. Consequently, inconsistent or erroneous interpretations can persist even when the underlying corpus is unchanged. Although provenance links support source checking, the paper does not specify how a correction discovered in a later query supersedes an earlier stored claim. Appendix D.1 illustrates that prior memory can misdirect subsequent search, but error propagation, conflict resolution, and recovery as memory accumulates are not systematically evaluated.
  3. The fixed-corpus evaluation omits a central operating condition of practical persistent-memory systems. A broad range of applications, including assistants accumulating ongoing interaction histories and agents using changing document collections, operate over evolving corpora; Section 3 itself acknowledges that new history may arrive continuously. Handling additions, revisions, and deletions is a core requirement for these applications, yet the paper does not specify how affected HET claims are invalidated or refreshed. This gap directly affects the practical significance of the efficiency results: maintaining valid memory introduces validation, invalidation, and refresh costs that the fixed-corpus experiments do not measure. Acknowledging the simplifying assumption and deferring knowledge maintenance to future work leaves this central requirement unresolved. A comparison under evolving corpora, measuring both answer quality and maintenance costs, is needed to establish whether MemSeeker’s reported advantages extend to these common operating conditions.
Questions for authors

NA

3 comments
AuthorsOpenReview ↗

Related work discussion and HET claim revision strategy

Thank you for highlighting these concerns about QTMC’s contribution, persistent correction, and corpus maintenance.

Q1: What does QTMC add beyond prior persistent-memory methods, and was HippoRAG2 omitted?

G-Memory and ReasoningBank study how experience from completed tasks can improve subsequent task execution, accumulating lessons about multi-agent collaboration and reasoning strategies, respectively. Their focus is “how to” perform tasks more effectively; QTMC studies how search itself can build memory of “what” a corpus contains and supports. As queries explore the corpus, the knowledge discovered becomes part of an accumulating semantic memory available to later queries. The connection is cross-task persistence, while the research objectives differ: improving task-solving behavior through experience versus constructing corpus knowledge through search. We have added this discussion to clarify QTMC’s positioning relative to both works.

HippoRAG2 was already evaluated using the official repository’s default configuration (Appendix C.1). Our “HippoRAG” label followed the repository name, but the citation incorrectly referred to HippoRAG1. We have corrected the label and citation to HippoRAG2 (ICML 2025); the evaluated implementation, configuration, and results remain unchanged.

Q2 and Q3: Conflict resolution, persistent correction, and knowledge updates

Our benchmarks already contain conflict-resolution and state-update scenarios: 235 MEMORA and 60 BEAM questions carry explicit annotations for conflicts and updates challenges within the supplied histories. We examine MemSeeker’s trajectories on this subset and find it can recognize outdated claims and follow newer evidence, because speculative retrieval returns raw records alongside stored claims, and agent infers the claim is stale if newer records supersedes it. The remaining gap is that correcting the current answer leaves the outdated claim available for subsequent retrieval.

These scenarios also challenge baselines with memory-update mechanisms. A-MEM uses an updated budget while retaining deleted deliverables, and Mem0 reports an intermediate rather than final budget. Such cases show that CRUD operations alone do not ensure correct maintenance: the system must determine which entity and attribute an update concerns and whether it supplements, supersedes, or retracts earlier information.

Building on MemSeeker’s existing ability to distinguish conflicting evidence, we add an explicit write-back mechanism so that discovered corrections persist. One added instruction asks the agent to cite or revise retrieved claims using C_n and C_n-revise. During summarization, a Known item citing an existing claim with evidence beyond its sources creates a new claim linked by derived_from, keeping both active. When the agent marks a superseded claim with C_n-revise, HET creates an active replacement linked by revision_of and excludes the old claim from retrieval while retaining it for provenance. We evaluated this extension on the annotated subset alongside the original system and baseline results:

MethodMEMORA (235)Tokens/q (k)BEAM (60)Tokens/q (k)
HippoRAG264.49123.3321.8882.91
Mem062.85280.4922.29269.15
A-MEM63.69299.9915.42295.47
SimpleMem61.08177.8938.12155.19
LightMem53.10220.2916.88172.36
MemSeeker w/o HET57.8185.6044.7960.14
MemSeeker original66.5272.3543.7552.44
MemSeeker revision extension69.2870.1947.7152.23

As the result shows, original MemSeeker already outperforms all external baselines on both subsets. Adding revision further improves performance by reducing distraction from outdated claims without increasing token consumption.

The trajectories show how these corrections carry forward. In the movie example, both versions follow the renewed preference for vampire movies, but the extension additionally retires the outdated claim and retrieves its replacement in four later queries. In the music example, both initially recognize the switch from shoegaze to indie pop and alternative rock. When neither retrieves the correcting record in a later query, the original again recommends shoegaze using the stale claim and supporting record, whereas the extension retrieves the replacement and follows the updated preference. Together, these cases illustrate how writing back a correction turns answer-time conflict resolution into reusable knowledge for subsequent queries.

Thanks for your response. I found two issues:

  1. MemSeeker with HET scores 66.52 on the Memora subset (235 questions), versus 57.81 without HET, a gain of 8.7 points. On the full 300 Memora questions in Table 1, HET scores 57.33 versus 58.12 without HET, a loss of 0.8 points. That means the remaining 65 questions would average about 24 with HET and about 59 without HET, i.e., HET would lose roughly 35 points on those questions.
  2. The original submission included a failure case in Appendix D.1 (p. 23, "Failure: wrong geographic scope"), in which earlier B-set claims led B-unseen to answer 1648, while B-stateless correctly answered 1625 (the gold answer). Section 5.3.1 referred to it (p. 8, l. 431), and my review cited this case. In the revision, Appendix E.4 keeps the three positive MuSiQue cases and Appendix E.2 adds two new positive cases, but the failure case and the reference to it are no longer present.
AuthorsOpenReview ↗

Typo for issue 1, modified for issue 2

Issue 1

Thank you for catching this inconsistency. We apologize for the reporting error: the correct MemSeeker w/o HET score on the 235-question Memora subset is 67.81, not 57.81. With this correction, original MemSeeker with HET scores 1.29 points lower than the variant without HET on this subset (66.52 vs. 67.81), rather than 8.71 points higher. We apologize for the confusion and appreciate your careful checking.

The conclusion is unchanged, that MemSeeker already has the ability of distinguishing the outdated claim and revision further improves this. Compared with MemSeeker w/o HET, original MemSeeker reduces token usage with slightly lower scores, indicating an efficiency benefit from persistent memory reuse. The revision extension further improves scores and reduces token usage, consistent with reduced distraction from outdated claims being excluded from subsequent retrieval.

The typo was introduced while reformatting the table. We have rechecked all the underlying evaluation results and confirmed that the table listed below has the correct value. We apologize for the error and the confusion it caused.

MethodMEMORA (235)Tokens/q (k)BEAM (60)Tokens/q (k)
HippoRAG264.49123.3321.8882.91
Mem062.85280.4922.29269.15
A-MEM63.69299.9915.42295.47
SimpleMem61.08177.8938.12155.19
LightMem53.10220.2916.88172.36
MemSeeker w/o HET67.8185.6044.7960.14
MemSeeker original66.5272.3543.7552.44
MemSeeker revision extension69.2870.1947.7152.23

issue 2

We add it to page 28-29, the last case study in Appendix E.4

Again, thank you for your valuable comments on our submission. As the discussion phase is approaching to its end, we would like to kindly confirm whether we have sufficiently addressed all of your concerns (or at least part of them). Should there be any remaining questions or areas requiring further clairification, please do not hesitate to let us know. If you are satisfied with our responses, we would greatly appreciate your consideration in adjusting the evaluation scores accordingly. We sincerely look forward to your feedback.

Reviewer_6ibJOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes a query-time memory construction paradigm for agent memory searches. That is, instead of constructing hierachical memory structures before query, or use stateless agent-as-query for memory searching at query time without constructing anything, this is a test-time-training-like mechanism which culmulates memory structures during memory queries.

Critical strengths

(1) This paradigm is interesting. Test-time construction of memory structures can be potentially useful in nowaday's agent systems.

(2) Experiments show good efficiency and effectiveness performance of the proposed paradigm compared to other memory paradigms.

Critical weaknesses

(1) The proposed improvement on CL, CL-Life and LongMemEval, as stated in Table 4, has no cross-query memory reuse chances. In these circumstances 84.5% fewer token usage is fully driven by better architecture design rather than the proposed reuse paradigm.

(2) The experiments where there are many query re-use, datasets that very prefer this property are selected. For example, In seciton 5.3.1, the musique A/B dataset, some evidence in problem B is exactly the same/is a subset/ is a super set of that of problem A.

I'd like to see more analysis on real/similar online settings where the similarity might be between CL CL-life Long Mem Eval and musique A/B dataset.

Questions for authors

Please see weakness.

2 comments
AuthorsOpenReview ↗

Cross-query persistence on general query sequences

Thank you for raising the distinction between within-query search efficiency and cross-query reuse. We first response the questions, then provide a persistence-disabled comparison and examine how historical knowledge is used in the general suite.

Q1: Are the token savings driven by the agent architecture rather than cross-query memory reuse?

A1: We agree that the token saving on these three benchmark comes fully from better architecture design. Improving the agent architecture is itself an explicit contribution: Contribution 2 on page 3 includes efficient agentic search. §5.2 identifies the LME gains without cross-query evidence reuse, and §5.3 distinguishes full-system performance from the contribution of reuse. Table 3 further ablates within-query context representation with persistence disabled, directly evaluating this architectural contribution.

Q2: Does evidence overlap in MuSiQue make knowledge reuse trivial?

We use MuSiQue specifically to examine the value of HET for cross-query knowledge reuse (§5.3.1) because its supporting-paragraph annotations let us control evidence overlap. Meanwhile, evidence overlap is measured at the paragraph level and does not imply identical information needs (Appendix B.3). For example, two questions in our ablation ask when the author of The Varieties of Religious Experience was born and when he died. Both use the same supporting paragraphs, but require different facts, 1842 and 1910. Successful reuse therefore depends on which conclusions HET retains and whether the agent can select and combine those relevant to the current question. While answering the birth-year question, the agent also recorded the author’s death year in its L0 working note, making that fact available for retrieval on the later question without rebundant search.

Based on this observation, we define evidence set relation to distinguish different patterns of knowledge reuse, and we provide some questions as example in Appendix E.3:

  • Equal: Reusing established knowledge to answer a different question over the same evidence, like the example stated above.
  • A-larger: Extracting relevant facts or intermediate conclusions from a broader prior investigation.
  • B-larger: Extending an existing understanding by combining retained knowledge with additional evidence.

This controlled setting lets us isolate the contribution of HET and clarify how retained knowledge benefits subsequent queries (Figure 3, and Table 10). The general-suite analysis provides complementary evidence under heterogeneous workloads.

Q3: More analysis on real online settings

We added a MemSeeker w/o HET comparison on the general suite and analyzed query streams without explicitly controlled evidence overlap. Table 2 includes 415 questions that require agentic search without HET, excluding single-query corpora. This population is selected independently of HET-side behavior.

The results show that HET’s benefit varies with observed reuse. The 134 questions with zero observed overlap shows historical claims are not invariably incorporated into the answer. Across nonzero-overlap groups, token savings range from 12.0% to 50.2%, with the largest savings in the highest-overlap group. Moreover, 82 questions requiring agentic search without HET are answered directly after initial retrieval with HET, reducing total token usage across these questions by 87.1%.

The trajectory analyses in Appendix E.2 illustrate how these benefits arise. In a Memora budgeting query, retained claims summarizing purchase records support a correct direct answer, reducing tokens from 74.3k to 22.7k. In an AMA trajectory-recall query, retained claims establish a partial event timeline and guide retrieval of missing actions, reducing additional search rounds from nine to one while yielding a correct answer.

Table 2 measures overlap from observed trajectories; MuSiQue provides complementary control through gold supporting-evidence relations. Both analyses support the same mechanisms: prior knowledge can directly support answers or focus subsequent search on missing evidence.

I'd like to thank the authors for the rebuttal replies.

I think for Q3, it's great to see further analysis w.r.t. token savings v.s. overlap Cq. This is intuitive and supports the authors' claim that query-time cache-like memory construction would save tokens, especially when queries have space- and time- locality. This shows a good intuition.

For Q2, I think this is an analysis of space/time locality properties of MuSiQue? This makes sense, but still I think when a dataset has time/spatial locality adding a cache-like mechanism would be helpful is not non-obvious. Anyway authors have shown a practical implementation of it.

Actually I'd like to see more analysis on different locality's impact on final results. I suggest authors look into some system papers on studying cpu cache performance for different types of tasks, and it might be possible to get some inspirations from there. E.g. different types of localities and how to measure performance w.r.t it, and also locality v.s. cache size, etc.

I'd like to raise my score from 2 to 3: weak accept if there are more analysis on these. Given the short rebuttal period I have increased my score, and I encourage authors to add more related experiments or analysis, (either before or after rebuttal deadline). I think they are not hard to figure out (just need some time to conduct).

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.