acceptodds
Under review as a conference paper at ICLR 2027

Just-In-Time Agent Memory with Runtime Agentic Research

Abstract

Memory is critical for AI agents. Many existing agent-memory systems follow an Ahead-of-Time (AOT) design, constructing memory before a specific request arrives. While this reduces online serving cost, such request-agnostic memory construction can discard fine-grained information that later becomes important. To address this limitation, we propose Just-In-Time Agent Memory (JAM), a trainable framework for query-conditioned context construction at runtime. A Memorizer preserves complete raw histories in a hierarchical page-store with compact navigational summaries, while a Researcher iteratively retrieves, inspects, and integrates evidence for each request. To train these memory-use behaviors, we introduce Memory-Gym, an evidence-grounded data synthesis pipeline covering nine task types across six domains, and optimize the Researcher through verified-trajectory supervised fine-tuning followed by Hint-guided Group Relative Policy Optimization. We demonstrate the effectiveness of JAM across a variety of benchmarks on agent memory and long-context processing, where it achieves stronger task performance than AOT-style memory systems while remaining substantially more efficient than prior trained agentic memory approaches. To support reproducibility and future research, we release our anonymized source code at https://anonymous.4open.science/r/Just-in-time-Agent-Memory-87D6.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Agent Memory, Runtime Agentic Research, Long-Context Reasoning, Reinforcement Learning for Agents

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_jTBuOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies how an agent can find and organize useful memories for a current query after accumulating a large interaction history. The authors propose JAM, which preserves the full history, builds a hierarchical workspace, and trains a Researcher to perform multiple rounds of retrieval and reading to construct query-relevant context. The main contributions are a Just-In-Time memory paradigm and a two-component framework that implements it: a Memorizer organizes historical information offline, while a Researcher conducts query-driven research at runtime. The paper reports stronger overall results than the evaluated baselines on LoCoMo, LongMemEval, NarrativeQA, and long-context HotpotQA, and demonstrates cross-domain transfer on LongCodeQA.

Critical strengths

The paper offers a meaningful perspective on memory system design. Future queries are often difficult to anticipate, and compressing history in advance may discard details that later become important. JAM provides a sensible response to this information-loss problem and offers useful insights into how computation can be allocated between memory writing and query processing. The experiments provide reasonably strong evidence of the design’s practical effectiveness. The method achieves strong results across several memory and long-context tasks, and the ablation studies examine the contributions of training and hierarchical organization.

Critical weaknesses

However, the paper has two main limitations. First, it lacks a controlled comparison with a strong multi-turn agentic RAG system. JAM’s workflow is closely related to agentic RAG, but the experiments do not establish its advantage under the same backbone, training data, and comparable online compute budgets. Second, the analysis and evaluation of long-term memory behavior remain limited. The paper does not directly assess how growing histories and other changes during continuous memory updates affect system performance. As evaluated, JAM is closer to a historical information retrieval system than a demonstrated mechanism for continual memory management.

Questions for authors

I would appreciate clarification on the following points:

  1. Have the authors compared JAM with a multi-turn agentic RAG system using the same backbone, the same Memory-Gym training data, and a comparable online compute budget? If so, what were the results?
  2. Is the intended scope query-conditioned context construction over a given history, or long-term memory management during ongoing interactions?
  3. Which component produces the final answer used for evaluation? The descriptions in the main paper and appendix are somewhat unclear on this point.
Reviewer_C3icOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes JAM, a just-in-time agent memory. Instead of compressing history before queries arrive, it keeps raw sessions in a hierarchical file workspace with LLM-written memos. For each query, an iterative open/search/browse agent Researcher assembles the context. With Qwen3.5-4B, JAM reports gains over AOT memory systems and 7–14B trained memory agents on LoCoMo, LongMemEval, NarrativeQA, and long-context HotpotQA, and it transfers to LongCodeQA.

Critical strengths
  1. The paper is generally well-motivated, easy to follow and read.
  2. The proposed method is simple yet effective on four widely used public benchmark.
  3. The open-source Memory-Gym is a reusable resource, which could be beneficial to the community.
Critical weaknesses
  1. The novelty and scope is relatively narrow. Maintaining raw text with its summary, and RL-trained agentic search are not new. The paper lacks discussion compared to the previous methods. Also, experiments are only limited to the QA task, agentic tasks such as tool call, computer use are not evaluated. So the “agent memory” framing is overstated.
  2. Under the paper’s own definition (raw storage plus query-time retrieval), standard RAG already counts as JIT memory, yet the paper positions JAM only against lossy AOT systems and does not clarify what it adds beyond agentic RAG methods such as IRCoT or Search-R1.
  3. Gains are not attributable to JIT itself. Without training, JAM underperforms RAG and full-context Vanilla on LME and HotpotQA, so the gains depend on Memory-Gym training, yet no baseline is trained on it for a controlled comparison. For the Hint-guided GRPO, hints built from the gold answer are injected into rollouts but removed at test time, wouldn't this introduce a train–test mismatch?
Questions for authors

see the weakness above.

1 comment

I appreciate the authors' efforts in reporting the new experimental results. I am inclined to keep my original score, though I would have no objection if the AC recommends this paper for acceptance.

Reviewer_TZxhOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes JAM (Just-In-Time Agent Memory), a framework that constructs query-specific memory contexts through runtime agentic research. JAM combines a Memorizer that organizes historical information into hierarchical workspaces with a Researcher that iteratively retrieves and integrates relevant evidence. The authors also introduce Memory-Gym, a synthetic training dataset, and optimize the Researcher using SFT and hint-guided GRPO. Experiments across four benchmarks demonstrate improvements over existing memory systems.

Critical strengths
  • The paper presents a well-motivated approach to overcoming information loss from request-agnostic memory compression through dynamic, query-conditioned retrieval.
  • Memory-Gym provides diverse training scenarios across multiple domains and task types, with encouraging cross-domain generalization results.
  • The experiments demonstrate strong performance across four benchmarks, supported by comprehensive ablation studies and efficiency analyses.
Critical weaknesses
  • The overall framework builds on existing hierarchical retrieval and agentic search techniques, making the methodological novelty somewhat limited.
  • The performance improvements may partly result from additional training data and optimization, rather than the proposed memory architecture itself, and the comparisons with trained baselines are not fully controlled.
  • The iterative Researcher introduces substantial online latency compared with conventional memory systems, potentially limiting practical deployment.
  • The evaluation does not sufficiently investigate scalability and robustness under continuously growing memory histories and frequent updates.
Questions for authors
  • Can the authors better isolate the contributions of the proposed architecture from those of Memory-Gym training and reinforcement learning?
  • How does JAM compare with existing agentic retrieval methods when using the same backbone, training data, and inference budget?
  • How does the performance and retrieval latency scale with significantly larger memory repositories and frequent memory updates?
  • How sensitive is the hint-guided GRPO training to the quality of generated hints, and how does the source-recall reward affect evidence precision?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.