acceptodds
Under review as a conference paper at ICLR 2027

Keep It in Mind: Implicit Memory via Latent-Space Knowledge Augmentation

Abstract

Successful trajectories provide language-model agents with reusable experience, but retaining and applying this experience across tasks remains challenging. External memory requires repeated retrieval and interpretation, while trajectory-based training embeds experience in model parameters, tying the incorporation of new experience to further model training. We introduce Knowledge-Augmenting Implicit Memory (KIM), which uses latent-space knowledge augmentation to retain learned behavioral adjustments in a memory separate from the backbone. Building on a fixed shared adapter, KIM learns local adjustments that favor successful continuations and jointly fits a memory field over their associated interaction states. This field integrates adjustments across trajectories to guide behavior in new states. During a new task, KIM reads a state-conditioned adjustment from the field and applies it through a low-rank intervention in the frozen backbone. Tool observations refresh the readout, guiding successive decisions without replaying source trajectories. With a memory field built from 3.6k successful trajectories selected by similarity to evaluation queries, KIM achieves 41.6% average accuracy across seven computational and knowledge-intensive reasoning benchmarks, compared with 30.8% for trajectory RAG using a 54k-trajectory retrieval bank. Applied to Tool-Star, KIM further raises average accuracy from 46.6% to 50.7%, extending its benefits to a model trained for multi-tool reasoning. Our code is available at https://anonymous.4open.science/r/KIM-63C8.

Primary area:
foundation or frontier models, including LLMs
Keywords:
language-model agents, implicit memory, latent-space knowledge augmentation, experience reuse, tool-augmented reasoning

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_BSdjOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes KIM, an implicit memory framework that converts successful interaction trajectories into state-conditioned low-rank parameter adjustments. KIM learns local adjustments for individual decision states, consolidates them into a shared memory field, and dynamically refreshes the resulting intervention during tool interaction. Experiments on seven reasoning benchmarks show substantial gains over the evaluated textual-memory methods and further improvements when KIM is applied to a tool-trained policy.

Critical strengths
  • The state-conditioned memory field provides a novel way to consolidate trajectory-specific experience into dynamically activated parameter adjustments.
  • KIM achieves substantial average gains across seven benchmarks and improves both instruction-tuned and tool-trained host policies.
  • The component ablations clearly examine the roles of general memory, contextual memory, the joint field, and state refresh.
Critical weaknesses
  • The main contextual memory bank is selected using the evaluation queries themselves, so the reported results do not cleanly establish generalization to unseen queries.
  • The comparison with textual-memory baselines does not isolate the contribution of the contextual field, since KIM additionally uses a general tool-use adapter trained on approximately 54k trajectories and the methods do not receive identical data access.
  • The evaluation is limited to 3B-scale backbones, while the absence of multiple training seeds leaves scalability and optimization stability unclear.
Questions for authors
  • How many of the 347 overlapping trajectories enter the final 3,609-trajectory contextual bank, and can the authors report results using an independently constructed, overlap-free bank?
  • Could the authors compare KIM with textual-memory methods using the same general tool-trained reference policy and the same non-overlapping trajectory pool, thereby isolating the contribution of the contextual field?
  • Does KIM retain its gains on 7B/8B backbones and across multiple training seeds?
4 comments
AuthorsOpenReview ↗

Response to BSdj (1/2)

Dear Reviewer BSdj,

Thank you for recognizing the design of the state-conditioned memory field and KIM’s gains on both instruction-tuned and tool-trained models. We address each concern below.

W1. The main contextual memory bank is selected using the evaluation queries themselves, so the reported results do not cleanly establish generalization to unseen queries.

The main experiment used 3,609 trajectories selected by similarity to the evaluation questions. To evaluate KIM on unseen queries, we exclude exact question overlaps and randomly sample 3,609 Tool-Star trajectories for RAG and KIM, without ranking them by similarity to the evaluation questions. All three methods use the same tool budget and judge. We report accuracy (%) on held-out GSM8K and HotpotQA questions and on four external benchmarks.

MethodGSM8K (100)HotpotQA (100)SVAMP (200)AIME25 (30)PopQA (200)FRAMES (824)Macro avg.
Tool-trained reference policy (no field)78.0037.0078.0010.0038.506.8041.38
Random-bank RAG + tool-use adapter75.0033.0082.006.6730.506.0738.87
KIM (random bank)82.0041.0087.5013.3344.0014.0846.99

KIM scores higher than both baselines on all six benchmarks. Its macro-average accuracy is 8.11 percentage points above adapter-equipped RAG on the same bank and 5.60 points above the tool-trained reference policy without the field.

W2.1. The comparison with textual-memory baselines does not isolate the contribution of the contextual field,

Table 2 holds the tool-trained reference policy and the memory bank fixed and varies only whether the field readout is enabled.

Field readoutKnowledge QA, 4 benchmarks (%)Math, 3 benchmarks (%)Seven-benchmark macro avg. (%)
Off20.5045.8931.38
On33.8051.8941.55

With the same tool-use adapter, enabling the field readout raises seven-benchmark macro-average accuracy from 31.38% to 41.55%. This isolates a gain of 10.17 percentage points from reading contextual memory.

W2.2. since KIM additionally uses a general tool-use adapter trained on approximately 54k trajectories

The main experiment uses roughly 54k historical trajectories to train the general tool-use adapter. To test the effect of training-pool size, we keep the RAG retrieval bank and training steps fixed and train its adapter on either the filtered large pool or a random subset of 3.6k trajectories. KIM (small training pool) uses only those 3.6k trajectories for adapter training, continuation and preference supervision, and memory writes.

Setting and methodGSM8KMATH500AIME24HotpotQA2WikiMuSiQueBamboogleMacro avg.
Full-bank RAG (no adapter)72.5049.506.6732.5020.008.5025.6030.75
KIM (main experiment, small bank)77.0062.0016.6743.5038.5018.0035.2041.55
Full-bank RAG + large-pool tool-use adapter68.5059.506.6734.5030.508.0026.4033.44
Full-bank RAG + small-pool tool-use adapter69.5057.006.6728.5026.508.5029.6032.32
KIM (small training pool)76.0060.5013.3344.5033.0014.0032.0039.05

When both training and memory writes are limited to 3.6k trajectories, KIM achieves 39.05% macro-average accuracy across the seven benchmarks, compared with 41.55% in the main experiment. KIM thus approaches its main result without access to the full 54k training pool.

W2.3. and the methods do not receive identical data access.

RAG and KIM use the same frozen step-3,700 tool-trained reference policy, the same random bank of 3,609 trajectories with exact evaluation overlaps excluded, and the same tool budget. RAG retrieves four trajectories per question as text examples; KIM writes the bank into its field and reads from it at the current interaction state.

MethodGSM8KMATH500AIME24HotpotQA2WikiMuSiQueBamboogleMacro avg. (%)
Random-bank RAG + tool-use adapter81.0064.506.6733.0034.509.5027.2036.62
KIM (random bank)76.5066.0010.0044.5040.5013.0032.0040.36

With matched data access and tool budgets, KIM scores 40.36% in seven-benchmark macro-average accuracy, above the 36.62% for adapter-equipped RAG on the same bank in the count-adjusted cross-machine summary. RAG scores higher on GSM8K; KIM scores higher on the other six benchmarks.

AuthorsOpenReview ↗

Response to BSdj (2/2)

W3.1. The evaluation is limited to 3B-scale backbones,

We extended the evaluation beyond Qwen2.5-3B to 8B backbones from two model families: Qwen3-8B and Llama-3.1-8B-Instruct. On each, we compared KIM with full-bank RAG on the original seven benchmarks.

BackboneMethodGSM8KMATH500AIME24HotpotQA2WikiMuSiQueBamboogleMacro avg. (%)
Qwen3-8BFull-bank RAG92.0080.0026.6749.5045.5021.0050.4052.15
Qwen3-8BKIM93.5085.0040.0051.0042.0021.0048.8054.47
Llama-3.1-8B-InstructFull-bank RAG82.0055.5010.0046.5031.0020.5044.8041.47
Llama-3.1-8B-InstructKIM82.0058.5013.3348.0044.0016.0051.2044.72

The advantage over full-bank RAG extends beyond the original 3B backbone: KIM leads by 2.32 percentage points on Qwen3-8B and 3.25 points on Llama-3.1-8B-Instruct in seven-benchmark macro-average accuracy.

W3.2. while the absence of multiple training seeds leaves scalability and optimization stability unclear.

We fix the 3B tool-trained reference policy, writing bank, training examples, and evaluation protocol. Only KIM’s initialization and training-example order vary. For each seed, we evaluate step 5,100 and compare its macro-average accuracy with the baseline reported in Table 2 with the field readout off (W2.1).

Method / training seedGSM8KMATH500AIME24HotpotQA2WikiMuSiQueBamboogleMacro avg. (%)
Field readout off (Table 2)70.0061.006.6731.0026.009.0016.0031.38
KIM, 20260725 (paper)77.0062.0016.6743.5038.5018.0035.2041.55
KIM, 2026072674.0060.5016.6745.5037.5017.0036.8041.14
KIM, 2026072778.5063.0016.6742.0040.5019.0033.6041.90

Across the three training seeds, KIM averaged 41.53% macro accuracy (sample SD: 0.38 percentage points).

Q1.1. How many of the 347 overlapping trajectories enter the final 3,609-trajectory contextual bank,

None. We excluded all 347 exact question overlaps before training and bank construction, so none entered the final bank of 3,609 trajectories. See our response to Reviewer e7qy (part 1/3, W1) for the separate experiment that added these 347 trajectories to a random bank.

Q1.2. and can the authors report results using an independently constructed, overlap-free bank?

W2.3 evaluates a bank of 3,609 independently sampled trajectories with exact evaluation overlaps excluded. KIM’s seven-benchmark macro-average accuracy is 40.36%, close to the 41.55% in the main experiment (W2.1), suggesting that overall performance on these benchmarks is not very sensitive to how the bank is constructed.

Q2. Could the authors compare KIM with textual-memory methods using the same general tool-trained reference policy and the same non-overlapping trajectory pool, thereby isolating the contribution of the contextual field?

W2.3 gives both methods the same tool-trained reference policy and overlap-free bank. KIM still outperforms adapter-equipped RAG (40.36% vs. 36.62% macro-average accuracy). W2.1 isolates the readout itself: enabling it raises macro-average accuracy by 10.17 percentage points with the adapter held fixed.

Q3.1. Does KIM retain its gains on 7B/8B backbones

In W3.1, we compared KIM with full-bank RAG on two 8B backbones. KIM retained its advantage on both, leading by 2.32 percentage points on Qwen3-8B and 3.25 on Llama-3.1-8B-Instruct in seven-benchmark macro-average accuracy.

Q3.2. and across multiple training seeds?

In W3.2, seven-benchmark macro-average accuracy ranged from 41.14% to 41.90% across three training seeds. All three exceeded the 31.38% reported with the field readout off in Table 2.

Thank you for encouraging us to test whether the gains hold across backbones and training seeds.

Best regards,
The authors

Thanks for the additional experiments. They address my main concerns, so I'll raise my score from 2 to 3.

AuthorsOpenReview ↗

Thank You for Your Feedback

Dear Reviewer BSdj,

Thank you very much for taking the time to revisit our response and reconsider your assessment. We greatly appreciate your constructive feedback and positive reassessment of our work. Your comments have helped us strengthen the generalization experiments, clarify the baseline comparisons, and improve the empirical evaluation and presentation of our work.

With sincere appreciation,
All Authors

Reviewer_e7qyOpenReview ↗
Rating
2
Confidence
3
Informativeness
3
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes Knowledge-Augmenting Implicit Memory (KIM), which converts successful agent trajectories into state-dependent low-rank interventions on a frozen backbone. Starting from a reference policy, KIM learns local adjustment vectors at decision boundaries and compiles them into a shared memory field. During inference, the current interaction state determines the adjustment, which is refreshed after tool observations. The paper reports improvements over textual memory baselines across seven reasoning benchmarks, as well as gains when augmenting Tool-Star.

Critical strengths
  1. Relevant problem and coherent design. Reusing experience without repeatedly inserting historical trajectories into the context is a worthwhile direction. The combination of local writing, memory compilation, and state-dependent readout provides a coherent framework for multi-turn tool interaction.

  2. Useful component ablations. Table 2 shows that the contextual memory improves over the shared-adapter-only reference policy, from 31.38% to 41.55% macro-average accuracy. Removing the joint field or state refresh also reduces performance. The paired bootstrap intervals provide additional support for these differences under the evaluated setting.

  3. Potential practical benefits. The compiled memory is compact, and field readout does not require scanning source trajectories. The improvement from 46.6% to 50.7% on Tool-Star also suggests that KIM can complement an already tool-trained policy.

Critical weaknesses
  1. Unclear training–evaluation separation. Appendix A reports 347 source trajectories whose questions overlap with the evaluation set, but does not clearly establish their exclusion from every KIM training stage. Different filtering rules for textual baselines further complicate the comparison.

  2. Insufficient evidence of OOD generalization. Evaluation queries participate in memory construction, and the update experiment uses within-dataset splits. These settings do not establish whether local experience transfers to unseen datasets without target-query-dependent selection.

  3. Unmatched baseline training. KIM receives additional trajectory supervision, whereas textual memory baselines primarily use the original backbone. Comparing against RAG with the same shared adapter and matched-data LoRA training would better isolate the memory mechanism’s contribution.

  4. Insufficiently isolated novelty. Local experiential learning and input-conditioned parameterized memory have precedents in DyPRAG [1] and MAC [2]. The paper should clarify what the joint memory field contributes beyond these existing ideas.

  5. Limited backbone coverage. Experiments focus on relatively early Qwen2.5-3B models. It remains unclear whether the gains persist on newer, stronger backbones or primarily compensate for limitations in small models’ reasoning and tool use.

[1] https://arxiv.org/abs/2503.23895 [2] https://arxiv.org/abs/2403.04317

Questions for authors
  1. Does the contextual memory improve performance on a completely held-out dataset?
  2. How does KIM compare with RAG using the exact same shared-adapter reference policy, source data exclusions, and interaction budget?
6 comments
AuthorsOpenReview ↗

Response to e7qy (1/3)

Dear Reviewer e7qy,

Thank you for recognizing KIM’s compact memory design and the value of our component ablations. We address each concern below.

W1. Unclear training–evaluation separation. Appendix A reports 347 source trajectories whose questions overlap with the evaluation set, but does not clearly establish their exclusion from every KIM training stage. Different filtering rules for textual baselines further complicate the comparison.

As reported in Appendix A, screening the Tool-Star trajectories identified 347 whose questions matched evaluation questions exactly. We excluded all 347 before bank construction and training in the main KIM experiment. The textual-memory baselines also excluded them before building their retrieval banks.

Only the random-mixing condition reintroduced the 347 overlapping trajectories. We started from an independently constructed bank with no exact question overlaps and replaced 347 randomly chosen trajectories with the 347 exact matches, keeping the bank size at 3,609. The table reports accuracy (%) for the main experiment and the random-mixing condition.

Experimental settingGSM8KMATH500AIME24HotpotQA2WikiMuSiQueBamboogleMacro avg.
KIM (main experiment, small bank)77.0062.0016.6743.5038.5018.0035.2041.55
KIM (random trajectory mixing)79.0065.0016.6745.0035.5014.5036.0041.67

The two settings achieve similar macro-average accuracy, with modest differences on individual benchmarks. We will add this mixing experiment to the revised appendix to clarify how we handled overlap with the evaluation questions at each training stage.

AuthorsOpenReview ↗

Response to e7qy (2/3)

W2.1. Insufficient evidence of OOD generalization. Evaluation queries participate in memory construction,

KIM does not build a separate memory for each evaluation query or question type. Our aim is to construct one memory field shared across tasks from a small set of successful Tool-Star trajectories.

Both the text-retrieval baselines and KIM draw on Tool-Star trajectories. The table compares the 53,624 source records remaining after excluding 347 known overlaps with the evaluation questions and the 3,609 trajectories selected for KIM’s memory using the evaluation queries.

Matched source benchmark(s)Tool-Star source records (share)Trajectories used to build KIM’s memory (share)
NuminaMath24,273 (45.27%)1,641 (45.47%)
GSM8K246 (0.46%)14 (0.39%)
GSM8K + NuminaMath6,481 (12.09%)417 (11.55%)
HotpotQA2,162 (4.03%)230 (6.37%)
2WikiMultiHopQA3,053 (5.69%)310 (8.59%)
WebWalker447 (0.83%)33 (0.91%)
SimpleDeepSearch3 (0.01%)1 (0.03%)
2WikiMultiHopQA + SimpleDeepSearch15 (0.03%)1 (0.03%)
Other16,944 (31.60%)962 (26.66%)
Total53,624 (100.00%)3,609 (100.00%)

“+” denotes a question matched to multiple datasets; each trajectory is counted once.

NuminaMath accounts for most Tool-Star source records (57.35%) and most trajectories used to build KIM’s memory (57.02%), including cross-dataset matches. NuminaMath is not one of our seven evaluation benchmarks. Four of those benchmarks do not appear among the identified trajectory sources: MATH500, AIME24, MuSiQue, and Bamboogle.

W2.2. and the update experiment uses within-dataset splits. These settings do not establish whether local experience transfers to unseen datasets without target-query-dependent selection.

We build a single KIM field from 3,609 randomly sampled source trajectories and evaluate it directly on four unseen benchmarks (R0). No target questions or feedback are used to select trajectories, write the memory, or retrain the policy. We compare KIM with the same tool-trained policy without a field and with RAG using that policy and the same trajectory bank.

External benchmark (questions)Tool-trained reference policy (no field)Random-bank RAG + tool-use adapter (R0 bank)KIM direct transfer (R0)KIM (R3, two-chain mean)R3 − R0 (pp)
SVAMP (200)78.00%82.00%87.50%87.50%0.00
AIME25 (30)10.00%6.67%13.33%13.33%0.00
PopQA (200)38.50%30.50%44.00%44.75%+0.75
FRAMES (824)6.80%6.07%14.08%14.93%+0.85
Four-benchmark macro avg.33.32%31.31%39.73%40.13%+0.40

The same KIM field transfers across all four unseen benchmarks: it outperforms both baselines on each, with 39.73% macro-average accuracy versus 33.32% for the reference policy and 31.31% for same-bank RAG. Three rounds of the paper’s lightweight updates on the source tasks yield a small further gain on these unseen benchmarks, bringing the two-chain mean to 40.13% (+0.40 percentage points).

W3. Unmatched baseline training. KIM receives additional trajectory supervision, whereas textual memory baselines primarily use the original backbone. Comparing against RAG with the same shared adapter and matched-data LoRA training would better isolate the memory mechanism’s contribution.

Adding the step-3,700 large-pool adapter to full-bank RAG leaves its four retrieved trajectories and their order unchanged. LoRA uses KIM's continuation examples, preference pairs and 5,100 updates. The last two rows match the frozen policy, overlap-free random bank (3,609 trajectories) and tool budget; RAG rates use the cross-machine summary rounded to benchmark counts.

MethodGSM8KMATH500AIME24HotpotQA2WikiMuSiQueBamboogleMacro avg. (%)
Full-bank RAG (no adapter)72.5049.506.6732.5020.008.5025.6030.75
Full-bank RAG + large-pool tool-use adapter68.5059.506.6734.5030.508.0026.4033.44
Standard LoRA (matched supervision)75.5060.003.3338.5037.5015.0036.0037.98
KIM (main experiment, small bank)77.0062.0016.6743.5038.5018.0035.2041.55
Random-bank RAG + tool-use adapter81.0064.506.6733.0034.509.5027.2036.62
KIM (random bank)76.5066.0010.0044.5040.5013.0032.0040.36

KIM outperforms LoRA with matched supervision (41.55% vs. 37.98% macro-average accuracy) and RAG with the same reference policy and bank (40.36% vs. 36.62%).

AuthorsOpenReview ↗

Response to e7qy (3/3)

W4. Insufficiently isolated novelty. Local experiential learning and input-conditioned parameterized memory have precedents in DyPRAG [1] and MAC [2]. The paper should clarify what the joint memory field contributes beyond these existing ideas.

KIM’s joint memory field can both equip a tool-use policy with experience from past interactions and augment the two methods you highlight, DyPRAG [1] and MAC [2]. Both methods already adapt parameters to the input. In the combined setting, we retain that adaptation and add an adjustment that KIM reads from past experience for the current interaction state. KIM fits the field jointly across historical states and their local adjustments, so the contribution of each write depends on its match to the current state and on its relation to the other stored states.

For each method, the first and third rows compare the baseline alone with the same baseline augmented by KIM’s joint field. The middle row uses local readout with the same historical writes and KIM checkpoint as the third, isolating the contribution of joint fitting.

Accuracy (%) on each benchmark; the final column gives the unweighted mean across all seven.

MethodGSM8KMATH500AIME24HotpotQA2WikiMuSiQueBamboogleMacro avg.
DyPRAG81.0062.5010.0027.5028.508.0012.0032.79
DyPRAG + local readout76.5063.0013.3329.5027.508.5020.0034.05
DyPRAG + joint field77.0064.5013.3335.5034.008.5029.6037.49
MAC66.5051.503.3327.5028.508.0012.0028.19
MAC + local readout66.5051.503.3329.5027.508.5020.0029.55
MAC + joint field66.5051.503.3335.5034.008.5029.6032.70

Adding KIM’s joint field raises seven-benchmark macro-average accuracy from 32.79% to 37.49% for DyPRAG (+4.70 percentage points) and from 28.19% to 32.70% for MAC (+4.51 points). Local readout of the same writes yields 34.05% and 29.55%, respectively, below the joint field in both cases. These comparisons show how KIM complements the two methods: even when the underlying method already adapts parameters to the input, an additional adjustment learned jointly from past interactions improves its performance.

[1] https://arxiv.org/abs/2503.23895

[2] https://arxiv.org/abs/2403.04317

W5. Limited backbone coverage. Experiments focus on relatively early Qwen2.5-3B models. It remains unclear whether the gains persist on newer, stronger backbones or primarily compensate for limitations in small models’ reasoning and tool use.

KIM retains its advantage over full-bank RAG on both 8B backbones: seven-benchmark macro-average accuracy is 2.32 percentage points higher on Qwen3-8B and 3.25 points higher on Llama-3.1-8B-Instruct.

BackboneMethodGSM8KMATH500AIME24HotpotQA2WikiMuSiQueBamboogleMacro avg. (%)
Qwen3-8BFull-bank RAG92.0080.0026.6749.5045.5021.0050.4052.15
Qwen3-8BKIM93.5085.0040.0051.0042.0021.0048.8054.47
Llama-3.1-8B-InstructFull-bank RAG82.0055.5010.0046.5031.0020.5044.8041.47
Llama-3.1-8B-InstructKIM82.0058.5013.3348.0044.0016.0051.2044.72

Q1. Does the contextual memory improve performance on a completely held-out dataset?

W2.2 tests transfer on four held-out external benchmarks. On SVAMP, for example, KIM answers 175/200 questions correctly (87.50%) without using SVAMP questions or feedback to build its memory, 9.50 percentage points above the tool-trained reference policy without the field (78.00%). The existing memory thus improves accuracy without task-specific retraining.

Q2. How does KIM compare with RAG using the exact same shared-adapter reference policy, source data exclusions, and interaction budget?

W3 matches the frozen tool-trained reference policy, overlap-free random bank and interaction budget. KIM scores 40.36% versus 36.62% for adapter-equipped RAG in seven-benchmark macro-average accuracy, so its advantage persists when both methods have access to the same trajectories.

We appreciate your questions about what the joint field adds to existing memory methods.

Best regards,
The authors

Reviewer_LdaYOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes a latent memory framework, called Knowledge-Augmenting Implicit Memory (KIM), that enables LLM agents to learn from the past successful trajectories implicitly without retrieving them as text. KIM first selects successful trajectories and learns state-specific behavioral adjustment vectors that increases the likelihood of successful actions. It then trains memory mapping from the agents' hidden-state representations of these adjustment vectors and use the mapping to dynamically adjust the weights given new contexts. Experiments across seven reasoning benchmarks show that KIM improves average accuracy from 30.8% with trajectory RAG to 41.6%, and further improves an existing tool-trained agent (Tool-Star) from 46.6% to 50.7%.

Critical strengths
  1. This paper provides a novel way of memory storage and self-improvement. Instead of storing the memory as text, this paper proposes to store the trajectories in latent space and use the latent memory to update the weights. This is a meaningful alternative to textual memory and static parameter-based learning.
  2. The memory field can incorporate new experiences by learning local adjustment vectors and updating ridge-regression sufficient statistics, without retraining the backbone or replaying all historical trajectories. The reported additional storage is only 4.89 MiB, compared with 1.09 GiB for trajectory RAG. This is particularly appealing for long-running agents.
  3. The experiments show large improvment compared with the RAG baseline.
Critical weaknesses
  1. Missing a strong, data-matched LoRA fine-tuning baseline. KIM introduces a complex pipeline involving local adjustment optimization, memory-field regression, and joint training. But the paper does not provide a clean comparison against conventional LoRA fine-tuning using identical successful trajectories, supervision, and compute budgets. A comparison with a simpler state-conditioned mixture of LoRA adapters trained end-to-end would be helpful for better understanding the advantage of this method.
  2. In the main experiments, the authors select 3,609 successful trajectories from a larger historical pool using semantic similarity to the evaluation queries. This means the memory has been constructed with advance knowledge of the evaluation questions. Although the evaluation answers are not directly supplied to the agent, this weakens the claim that KIM generalizes to unseen tasks through accumulated experience. Moreover, KIM's general tool-use adapter and contextual supervision also leverage the larger 54k trajectory pool. I would want to see experiments where memory is constructed independently of the evaluation questions, followed by testing on genuinely unseen task distributions.
Questions for authors

Please address the weakness I flagged above.

4 comments
AuthorsOpenReview ↗

Response to LdaY (1/3)

Dear Reviewer LdaY,

Thank you for recognizing KIM’s latent memory mechanism and its ability to incorporate new experience without retraining the backbone. We address each concern below.

W1.1. Missing a strong, data-matched LoRA fine-tuning baseline. KIM introduces a complex pipeline involving local adjustment optimization, memory-field regression, and joint training. But the paper does not provide a clean comparison against conventional LoRA fine-tuning using identical successful trajectories, supervision, and compute budgets.

We compare KIM with three static fine-tuning methods. Standard LoRA learns a low-rank weight update. DoRA [1] updates the direction of the weights with LoRA and learns their magnitudes separately. AdaLoRA [2] allocates rank during training according to the importance of individual update components, allowing different weight matrices to retain different numbers of update directions. All three use a fixed weight update at inference. KIM instead combines low-rank updates according to the current interaction state and refreshes the combination after each tool response.

All four methods start from the same tool-trained reference policy, use the same successful trajectories, continuation examples, preference pairs, and number of updates, and are evaluated on the same questions. The table reports accuracy (%) on each benchmark and their unweighted mean.

MethodGSM8KMATH500AIME24HotpotQA2WikiMuSiQueBamboogleMacro avg. (%)
Standard LoRA (matched supervision)75.5060.003.3338.5037.5015.0036.0037.98
DoRA77.5065.006.6744.5035.0014.0033.6039.47
AdaLoRA76.5064.006.6742.5037.0015.0035.2039.55
KIM (main experiment, small bank)77.0062.0016.6743.5038.5018.0035.2041.55

KIM leads AdaLoRA, the strongest static baseline, by 2.00 percentage points in macro-average accuracy. Its largest gains are on AIME24 (10 points) and MuSiQue (3 points). Neither benchmark appears among the identified sources of the Tool-Star trajectories. These results suggest that composing local updates learned from past trajectories to suit the current interaction state helps KIM transfer experience to these tasks more effectively than a fixed weight update. We discuss trajectory sources and generalization in detail in W2.

[1] https://arxiv.org/abs/2402.09353

[2] https://arxiv.org/abs/2303.10512

W1.2. A comparison with a simpler state-conditioned mixture of LoRA adapters trained end-to-end would be helpful for better understanding the advantage of this method.

Both the state-conditioned LoRA mixture and KIM can change the weight update after a tool response. The mixture learns the coefficients of its LoRA operators directly from the current state, end to end. KIM first learns a local update for each historical interaction state, then fits a shared memory field to those updates. We compare the two using the same state features, 16 rank-4 operators, and the same refresh points, with the reference policy, supervision sequence, and 5,100 updates matched as in W1.1.

MethodGSM8KMATH500AIME24HotpotQA2WikiMuSiQueBamboogleMacro avg. (%)
State-conditioned LoRA mixture76.0063.003.3342.0038.0016.5031.2038.58
KIM (main experiment, small bank)77.0062.0016.6743.5038.5018.0035.2041.55

KIM outperforms the state-conditioned LoRA mixture in macro-average accuracy (41.55% vs. 38.58%), with the largest gains on AIME24 and Bamboogle. Learning local updates from past trajectories, rather than mixture weights directly end to end, may make those updates easier to reuse at similar steps in new problems.

AuthorsOpenReview ↗

Response to LdaY (2/3)

W2.1. In the main experiments, the authors select 3,609 successful trajectories from a larger historical pool using semantic similarity to the evaluation queries. This means the memory has been constructed with advance knowledge of the evaluation questions.

Our aim is to show that a small set of successful Tool-Star trajectories suffices to build a single memory field shared across tasks, without constructing a separate memory for each question type. The table compares the sources of the 53,624 Tool-Star records remaining after excluding 347 known overlaps with the evaluation questions and the 3,609 trajectories written into KIM’s memory.

Matched source benchmark(s)Tool-Star source records (share)Trajectories used to build KIM’s memory (share)
NuminaMath24,273 (45.27%)1,641 (45.47%)
GSM8K246 (0.46%)14 (0.39%)
GSM8K + NuminaMath6,481 (12.09%)417 (11.55%)
HotpotQA2,162 (4.03%)230 (6.37%)
2WikiMultiHopQA3,053 (5.69%)310 (8.59%)
WebWalker447 (0.83%)33 (0.91%)
SimpleDeepSearch3 (0.01%)1 (0.03%)
2WikiMultiHopQA + SimpleDeepSearch15 (0.03%)1 (0.03%)
Other16,944 (31.60%)962 (26.66%)
Total53,624 (100.00%)3,609 (100.00%)

“+” denotes a question matched to multiple datasets; each trajectory is counted once.

NuminaMath accounts for most Tool-Star source records (57.35%) and most trajectories used to build KIM’s memory (57.02%), including cross-dataset matches. NuminaMath is not one of our seven evaluation benchmarks. Four of those benchmarks do not appear among the identified trajectory sources: MATH500, AIME24, MuSiQue, and Bamboogle.

W2.2. Although the evaluation answers are not directly supplied to the agent, this weakens the claim that KIM generalizes to unseen tasks through accumulated experience.

To test whether KIM relies on query-based selection, we compare the original trajectory bank, a random bank of the same size (3.6k trajectories), and the full bank of roughly 54k eligible trajectories.

Accuracy (%) on the same seven benchmarks:

Setting and methodGSM8KMATH500AIME24HotpotQA2WikiMuSiQueBamboogleMacro avg.
KIM (main experiment, small bank)77.0062.0016.6743.5038.5018.0035.2041.55
KIM (full bank)76.0063.0010.0046.0040.0013.5030.4039.84
KIM (random bank)76.5066.0010.0044.5040.5013.0032.0040.36
Full-bank RAG72.5049.506.6732.5020.008.5025.6030.75

KIM achieves its highest macro-average accuracy with the bank construction method used in the main experiment (41.55%), compared with 40.36% for the random bank and 39.84% for the full bank. The three settings span just 1.71 percentage points, suggesting that overall performance on these seven benchmarks is not very sensitive to how the bank is constructed.

AuthorsOpenReview ↗

## Response to LdaY (3/3)

W2.3. Moreover, KIM's general tool-use adapter and contextual supervision also leverage the larger 54k trajectory pool.

Following Tool-Star, we use the full 54k trajectory pool to train the general tool-use adapter so that the agent learns from a wider range of tool calls. To test whether the adapter can be trained on fewer trajectories, we sample 3,609 trajectories at random from the filtered training pool of 50,756. Both adapter-equipped RAG variants retrieve from the full bank; their adapters are trained on the large or small pool for the same number of steps. KIM (small training pool) uses only those 3.6k trajectories for continuation and preference supervision and writes them into memory.

Training pool size (accuracy, %).

Setting and methodGSM8KMATH500AIME24HotpotQA2WikiMuSiQueBamboogleMacro avg.
Full-bank RAG (no adapter)72.5049.506.6732.5020.008.5025.6030.75
KIM (main experiment, small bank)77.0062.0016.6743.5038.5018.0035.2041.55
Full-bank RAG + large-pool tool-use adapter68.5059.506.6734.5030.508.0026.4033.44
Full-bank RAG + small-pool tool-use adapter69.5057.006.6728.5026.508.5029.6032.32
KIM (small training pool)76.0060.5013.3344.5033.0014.0032.0039.05

Trained on 7.1% of the filtered pool, KIM achieves 39.05% macro-average accuracy across the seven benchmarks, compared with 30.75% for RAG without a tool-use adapter. RAG’s macro-average accuracy with the small training pool (32.32%) remained close to that with the large pool (33.44%).

W2.4. I would want to see experiments where memory is constructed independently of the evaluation questions, followed by testing on genuinely unseen task distributions.

RAG follows the paper’s main-table setup: the unmodified backbone retrieves four trajectories per question from the full bank of 50,781. RAG + tool-use adapter and KIM share the same tool-trained reference policy and a bank of 3,609 trajectories sampled at random after excluding exact question overlaps. The former retrieves four trajectories from this bank; KIM builds its memory from all 3,609 trajectories. The SVAMP and PopQA entries for plain RAG are inferred from whole-percentage results. We report accuracy (%) on held-out GSM8K and HotpotQA questions and four external benchmarks; parentheses give the number of questions.

MethodGSM8K (100)HotpotQA (100)SVAMP (200)AIME25 (30)PopQA (200)FRAMES (824)Avg.
Qwen2.5-3B-Instruct (no adapter)15.007.0015.003.336.501.948.13
Full-bank RAG (no adapter)67.0032.0076.006.6725.008.0135.78
Tool-trained reference policy (no field)78.0037.0078.0010.0038.506.8041.38
Random-bank RAG + tool-use adapter75.0033.0082.006.6730.506.0738.87
KIM (random bank)82.0041.0087.5013.3344.0014.0846.99

Without task-specific retraining, KIM scored higher than both adapter-equipped RAG and the tool-trained reference policy without the field on all six benchmarks.

Q1. Please address the weakness I flagged above.

W1 match the training examples and update count. KIM reaches 41.55% macro-average accuracy, above both the strongest static baseline, AdaLoRA (39.55%), and the state-conditioned LoRA mixture (38.58%).

W2 test an independently sampled bank. KIM scores 40.36% across the original seven benchmarks and answers 175/200 SVAMP questions correctly (87.50%) without retraining or rebuilding the bank. The same field thus transfers to a task not used to construct it.

Thank you for suggesting the matched baselines and the transfer tests on unseen tasks.

Best regards,
The authors

AuthorsOpenReview ↗

Follow-up response to LdaY

Dear Reviewer LdaY,

Thank you again for your thoughtful and constructive review. As the discussion period is drawing to a close, we understand that this is a busy time and that you may not yet have had the opportunity to revisit our recent updates. Over the past few days, we have continued investigating your concerns about strong baselines and generalization to unseen tasks, and we would be grateful for the opportunity to discuss our new findings with you.

We wanted to investigate whether KIM's advantages could generalize to other task types when its memory field was trained only on mathematical reasoning and question-answering trajectories. We therefore chose code generation, using the widely adopted MBPP benchmark and its extended test suite, MBPP+, to compare KIM with LoRA, AdaLoRA, and a state-conditioned LoRA mixture. All four methods use their existing checkpoints without code-specific retraining. They directly generate code for the same 378 problems, without tool calls, and are evaluated by executing the test cases.

Pass rates (%):

MethodMBPPMBPP+
LoRA70.9060.85
AdaLoRA71.9661.38
State-conditioned LoRA mixture72.2261.38
KIM73.8162.43

KIM achieves the highest pass rate on both evaluations. These results provide additional evidence of transfer from mathematical reasoning and QA experience to code generation: the existing memory field can be reused without code-specific retraining, while retaining an advantage over static fine-tuning and an end-to-end state-conditioned mixture.

Together with our earlier experiments, these results address your two main concerns. For the baseline and methodological contribution questions, we added LoRA, DoRA, AdaLoRA, and state-conditioned mixture comparisons with matched reference policies, training data, supervision, and update counts, examining the benefit of learning local experiential updates and consolidating them into a memory field. For generalization and data access, we evaluated independently sampled memory banks, unseen questions and external benchmarks, and training restricted to 3,609 trajectories, testing reliance on evaluation-query-based selection and the larger training pool.

These findings further highlight KIM's central contribution: consolidating local interaction experience into a shared memory field that supplies parameter adjustments according to the current state, without repeatedly inserting historical trajectories into the context. The MBPP and MBPP+ results further support extending this form of experience reuse to different task types.

We would greatly value your thoughts on these new results and would be happy to discuss or clarify anything further during the remaining time. We sincerely hope to earn your further support so that this work can reach a broader readership and contribute to the community's understanding of agent memory and learning from experience. Thank you again for your time and for the valuable suggestions that have helped us strengthen this work.

Best regards,
The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.