- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper addresses the problem of how the current agent systems reuse past experience to improve future performance. Most existing designs curate memory at write time: once a task is completed, its trajectory is distilled into a fixed artifact, such as a reflection, workflow, skill, or reasoning strategy, that is later retrieved by similarity. This forces the system to decide what is worth remembering before the future query is known, irreversibly discarding information and producing a query-independent summary that must serve many possible downstream tasks. Learning such a write-time curator is also difficult because the value of a storage decision may only become apparent when a relevant query arrives, potentially many tasks later, creating a long-horizon credit-assignment problem.
Instead, this study develops a JITMem strategy. It retains raw trajectories and defers curation until read time, when the current task is known. Given the retrieved traces and the new task, a memory curator synthesizes a compact, task-adaptive payload tailored to the immediate need. Because this payload is consumed on the same task, the curator can be trained directly from immediate task success, avoiding delayed utility signals and the need to artificially group related tasks.
Their experiments show that across ALFWorld, WebShop, and τ 2-bench, Just-in-Time Memory (JITMEM) consistently outperforms no-memory agents as well as heuristic and learned write-time memory methods, improving over the strongest baseline by 16.2, 16.3, and 3.9 absolute success-rate points, respectively. Notably, even an untrained curator is already competitive with or surpasses these baselines, showing that task-adaptive read-time curation itself is a major source of the gain; training the curator further compounds the improvement.
The study is interesting and timely, and based on my view, it does make progress on the agent handling and management although the curator may also be costly in its own which may need to be further examined in the discussion and evaluation.
Critical strengths
S1. The study poses a critical problem of the current agent system, that is, most existing designs curate memory at write time and distill the trajectory into a fixed artifact, such as a reflection, workflow, skill, or reasoning strategy, and later retrieved them by similarity. But such a strategy has a fundamental problem: The value of a storage decision may only become apparent when a relevant query arrives, potentially many tasks later, creating a long-horizon credit-assignment problem. This observation is good.
S2. The study works out a new strategy, called JITMem, which retains raw trajectories and defers curation until read time, when the current task is known. Given the retrieved traces and the new task, a memory curator synthesizes a compact, task-adaptive payload tailored to the immediate need. Because this payload is consumed on the same task, the curator can be trained directly from immediate task success, avoiding delayed utility signals and the need to artificially group related tasks.
S3: The performance study on three benchmark datasets, ALFWorld, WebShop, and τ 2-bench, shows JITMEM consistently outperforms no-memory agents as well as heuristic and learned write-time memory methods, improving over the strongest baseline 16.2, 16.3, and 3.9 absolute success-rate points, respectively. Notably, even an untrained curator is already competitive with or surpasses these baselines, showing that task-adaptive read-time curation itself is a major source of the gain; training the curator further compounds the improvement. This is very good progress.
Critical weaknesses
W1. The design is interesting. But when it defers curation until read time, when the current task is known. It may raise two problems: (1) how could you retrieve the previous traces that need to be curated now correctly and accurately? When the traces passed for quite some time, to correctly retrieve such traces may itself becomes a problem. This seems to be a concern, and (2) the cost of curation.
W2: Given the retrieved traces and the new task, a memory curator synthesizes a compact, task-adaptive payload tailored to the immediate need, this curation itself could be costly and nontrivial, which may need to be counted and joining into the cost study more seriously.
W3: How to curate in time, by following what standard, could be a theme of discussion. I also wonder whether there could be cases that the same retrieved cases should be curated differently based on different usages in later cases. What should be the standard and most efficient strategy to accomplish such a task.
Overall, the study is interesting and makes a good contribution to the generation of agent memory. I am a little on the borderline between weak accept and weak reject, depending on how the rebuttal could successfully address my concern above.
Questions for authors
I would like to see how the authors would address my concerns raised in the weakness. I am open for in-depth discussion of the paper.
5 comments
Authors' Response to Reviewer fGWu (Part 2)
W3: How to curate in time, by following what standard, could be a theme of discussion. I also wonder whether there could be cases that the same retrieved cases should be curated differently based on different usages in later cases. What should be the standard and most efficient strategy to accomplish such a task.
We thank the reviewer for this question.
Standard. In JITMEM, the curation standard is not hand-designed but learned from the task reward. Since the curated payload is applied to the same task it was produced for, we train the curator with GRPO using the task success of the executor as the reward.
Strategy. Curating the same retrieved trajectory adaptively based on the purpose of each later task is the design of JITMEM. Specifically, the curator reads the retrieved traces together with the current task, and then synthesizes a compact, task-conditioned payload. Because the curator knows the current task, it extracts the specific segments useful for that task, so the same stored trajectory can yield a different payload for each later tasks. This strategy parallels the cognitive-science view that episodic memory is reconstructive rather than replayed, with retrieval shaped by current goals and cues [1]. This discussion is in lines 50 to 84 of the submission.
Our case study in Figure 3 and Section 4.3 shows that the same retrieved trajectory is curated differently for two later tasks by JITMEM. Both tasks retrieve the trajectory "put a cool egg in microwave", a sequence of steps that includes a state-transition step (e.g., cooling an object) as well as object localization and placement steps. Each later task draws strategies from different steps. For "put a hot potato in fridge", the curator focuses on the state-change step and specializes it into guidance for heating the potato with a microwave. For "put a newspaper in sofa", it instead emphasizes the placement steps, curating guidance such as verifying the target location.
Reference
[1] Schacter and Addis. The cognitive neuroscience of constructive memory: remembering the past and imagining the future
A gentle reminder that the deadline for the author–reviewer discussion is approaching.
Dear Reviewer fGWu,
Thank you for your constructive feedback, which helped us improve our paper. As the discussion period is closing soon, we would like to follow up and see if the Reviewer has had a chance to consider our response. We have provided additional results and clarifications in our response. Please let us know if we should include anything further in the revised draft. We are more than happy to clarify if anything is unclear.
Best regards,
Authors
Authors' Response to Reviewer fGWu (Part 1)
We thank the reviewer for their thoughtful review and are glad that you found our work "interesting and makes a good contribution to the generation of agent memory". We address each comment below.
We understand the concern to be that, as time passes, the bank accumulates more traces, so the relevant ones must be found among more and more candidates. Three properties of JITMEM address this.
First, retrieval does not depend on when a trace was stored. BM25 ranks stored trajectories only by how well their task descriptions match the current task, and the bank never discards entries, so an old trace is as retrievable as a recent one.
Second, a larger bank has not hurt performance in our experiments. In Table 5 of Section 4.2, we show that pre-filling the test bank with 100 additional trajectories from the training set changes SR by at most 1.3 points.
Third, read-time curation tolerates imperfect retrieval. The curator reads all retrieved trajectories together with the current task and first identifies which of them are relevant.
Regarding the choice of retriever, we follow prior work SkillOS [1], so that the comparison is fair and the performance gain can be attributed to task-adaptive read-time curation rather than to a better retrieval method. All methods use the same retriever BM25, with the current task description as the query and past task descriptions as the keys. We acknowledge that BM25 is a simple choice, and an embedding-based or model-based retriever could perform better. However, we chose it to keep the experimental setup fair, and we note this as a limitation in the conclusion of our submission.
We thank the reviewer for this point. We agree that curation cost should be counted explicitly. Table 11 below gives a cost breakdown of the curator and the executor. Costs are computed from the token counts in Table 12.
All experiments are on ALFWorld with a Qwen3-8B curator and a GPT-5.4 executor. Costs are based on \0.2per1MinputandoutputtokensforQwen3−8B[2],and2.5 per 1M input tokens and \$15 per 1M output tokens for GPT-5.4 [3].
Table 11. Cost breakdown per task (USD) on ALFWorld with Qwen3-8B curator and GPT-5.4 executor. Costs are computed from the token counts in Table 12.
Table 12. Token breakdown per task (K) on ALFWorld with Qwen3-8B curator and GPT-5.4 executor. Columns marked with * are reported in Table 4 of the submission.
Reference
[1] Ouyang et al. SkillOS: Learning Skill Curation for Self-Evolving Agents
[2] https://pricepertoken.com/pricing-page/model/qwen-qwen3-8b
[3] https://developers.openai.com/api/docs/models/gpt-5.4