acceptodds
Withdrawn from ICLR 2027

Picorer: Agentic Memory, One Clue at a Time

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Agentic memory gives LLM agents control over how past experience is organized, retrieved, and used. Using this experience for a task requires deciding what information to acquire and how separate findings fit together. As the agent reads, newly discovered facts can change what it needs to seek next. Effective memory acquisition therefore demands adaptive exploration while preserving accumulated evidence. We present Picorer, a memory harness that acquires evidence one clue at a time. Its LLM policy revises a working state to guide search, while an independent ledger retains inspected passages for answering. Under at most eight search calls per query, Picorer raises MemoryAgentBench Overall from 41.60% to 49.17% with GPT-4o-mini, outperforming HippoRAG-v2 and MemGPT. With GPT-4.1-mini it leads the reproduced BEAM baselines at 72.52% and 60.49% on the 100K- and 10M-token archives, and with GPT-5-mini it reaches 90.00% on LoCoMo. Controlled interventions on multi-hop fact consolidation show that completing supporting relations yields larger accuracy gains than partial progress. Further reading can still improve accuracy after an independent model judges the evidence sufficient.

Primary area:
other topics in machine learning (i.e., none of the above)
Keywords:
Agent Memory, LLM

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_chbMOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes Picorer, an agentic memory retrieval framework. The main design is to separate a revisable working state from an accumulated evidence ledger. The working state keeps currently established facts and missing information, and is used to guide the next search/read action. The evidence ledger keeps the source passages that have already been read and provides them to the final answer model.

The paper evaluates Picorer on MemoryAgentBench, BEAM, and LoCoMo. It also includes ablations and additional analysis on multi-hop retrieval, especially about cumulative evidence retention, complete supporting chains, and whether the model can correctly judge when the retrieved evidence is already sufficient.

Critical strengths

1. The paper has relatively extensive empirical analysis. Compared with many system papers that only report final benchmark numbers, this paper tries to analyze what happens during the retrieval process. In particular, the analysis on partial support versus complete support chain is useful for understanding the behavior of multi-hop retrieval.

2. The ablation studies reveal some concrete behavior of the system. The paper shows that the structured note mainly reduces repeated retrieval, while cumulative retention of previously read evidence has a much larger effect on multi-hop accuracy. I think this analysis is useful even though it also raises questions about the actual novelty of the proposed method.

Critical weaknesses

1. My main concern is the limited novelty of the method. The major principles used in Picorer, such as iterative retrieval, adapting the next retrieval based on intermediate results, and keeping information obtained from previous retrieval steps, have already been explored in prior multi-hop and agentic retrieval systems.

The most explicit new component seems to be the structured Established/Missing working note. However, according to the paper's own ablation, removing this note almost does not affect the final accuracy on the multi-hop fact consolidation task. Its major effect is reducing repeated source accesses. This makes the note look more like an engineering mechanism for avoiding redundant retrieval rather than a major algorithmic contribution.

The larger performance effect comes from keeping previously read evidence instead of only keeping the latest read. However, cumulative retention of previous evidence seems to be a rather natural design choice for an iterative multi-hop retrieval system. Therefore, I have difficulty seeing a sufficiently strong methodological novelty beyond engineering refinement of existing ideas.

2. The current experiments do not fully isolate where the gain comes from. The abstract emphasizes the comparison with HippoRAG-v2, but this comparison is not under a fully matched experimental setting compared with other controlled baselines.

Also, the w/o note ablation still keeps most of the Picorer framework, including the search/read loop, source information, budget information, and cumulative evidence ledger. Therefore, it does not answer a simpler question: whether a basic iterative retrieval system with query reformulation and cumulative passage retention can already achieve similar performance.

Without such a baseline, it is difficult to tell whether the gain is really from the proposed framework or mainly from standard iterative retrieval plus keeping all previously retrieved evidence.

Questions for authors
  1. Can the authors provide a controlled comparison with HippoRAG-v2 under the same backbone, corpus, retrieval budget, and evaluation setting, since the paper emphasizes improvement over HippoRAG-v2?
  2. Can the authors compare against a simpler baseline that only performs iterative query reformulation/search and cumulatively keeps the retrieved passages, without the Established/Missing note and other Picorer-specific state design? I think this baseline is important to understand how much of the gain actually comes from the proposed framework.
Reviewer_t3mEOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes Picorer, which separates a revisable working state for iterative retrieval from an independent ledger that retains passages read by the agent for final answering. The authors evaluate it on MemoryAgentBench, BEAM, and LoCoMo with up to eight search calls per question, reporting improvements in some settings. Ablations and evidence interventions examine the roles of individual components, the completeness of supporting evidence, and the model’s judgments of evidence sufficiency.

Critical strengths
  1. The paper addresses a practical issue: repeatedly updating a working note can discard information still needed for the final answer. Separating the working state from retained source evidence allows the agent to revise its understanding while preserving material it has read. This is a reasonable design with potential value for agents that need to compress their working context.

  2. On MemoryAgentBench, Picorer achieves the highest Overall score among the GPT-4o-mini methods listed in the paper: 49.17%, compared with 42.20% for the long-context baseline and 41.60% for HippoRAG-v2 (Table 1). This supports its overall effectiveness in this evaluation setting and is the main empirical strength in my assessment.

Critical weaknesses
  1. The technical novelty is unclear. Existing methods already combine compressed memory with retaining or revisiting source text. Picorer explicitly separates retrieval control from evidence retention, but the new capabilities or substantive improvements this adds are not clear enough to support acceptance.

  2. The experiments do not directly establish how much of the ledger’s benefit comes from recovering information lost during updates to the working note, or how consequential such losses are. This remains unclear despite the ablations, evidence interventions, and fixed-source comparisons of summaries and source text. Different retrieval procedures and Picorer’s higher query token usage relative to SimpleMem (Table 8) also make it difficult to separate the effects of the proposed mechanism, more extensive evidence input, and additional computation.

Questions for authors
  1. Compared with related methods discussed in the paper, such as ReadAgent and General Agentic Memory (GAM), what is Picorer’s key technical contribution in updating the working state and retaining evidence from inspected sources? What new capabilities or substantive improvements do these differences provide?

  2. The ledger retains evidence already read, but omissions or errors in the working note may still affect subsequent retrieval or stopping decisions. Could the authors clarify which problems this design can address and provide more direct evidence of how common information loss during note updates is in the evaluated tasks, and how much the ledger helps in these cases?

  3. Based on the current evidence, could the authors clarify how the proposed separation and additional computation during querying each contribute to the performance gains, and explain the associated trade-off between performance and cost?

Reviewer_YbGXOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes Picorer, an agentic memory framework that enables adaptive, multi-step retrieval from long-term memory. The key idea is to separate a revisable working state, which guides subsequent retrieval decisions, from an independent evidence ledger that preserves previously retrieved information for answering. The authors evaluate Picorer on MemoryAgentBench, BEAM, and LoCoMo, reporting improvements over several existing memory systems. Additional experiments investigate the effects of evidence retention, structured working states, and evidence completeness on multi-hop question answering.

Critical strengths
  1. Practical and relevant problem. The paper addresses an important challenge in agentic memory: adaptively retrieving and integrating information across multiple steps to answer complex queries.

  2. Extensive empirical analysis. The paper evaluates multiple memory benchmarks and provides detailed ablation studies and evidence-level analyses to investigate the effects of evidence retention and completeness on answer quality.

Critical weaknesses
  1. Limited methodological novelty. Picorer mainly combines existing techniques, including iterative retrieval, structured working notes, and evidence accumulation. Its distinction from standard ReAct-style retrieval agents remains unclear.

  2. Missing important baselines. The evaluation omits representative memory systems such as Mem0. Without comparisons against these fundamental baselines, it is difficult to establish Picorer's advantages over existing memory approaches.

  3. Inconsistent backbone comparisons. Many baselines are evaluated only with smaller models, while stronger backbones such as GPT-5-mini and Qwen3.6-27B are primarily used for Picorer. This makes it difficult to separate improvements from the proposed method and those from stronger models.

  4. Unclear experimental fairness. The paper groups BM25 under GPT-4o-mini, but does not sufficiently clarify how the retrieval and answering components are configured across baselines. The fairness of the comparisons in terms of retrieval budgets, model usage, and computational costs needs further justification.

  5. Limited novelty of empirical findings. The finding that complete evidence chains improve multi-hop reasoning is largely expected, and the paper does not propose a new algorithm to exploit this observation.

Questions for authors

See Weakness

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.