Keep It in Mind: Implicit Memory via Latent-Space Knowledge Augmentation

Successful trajectories provide language-model agents with reusable experience, but retaining and applying this experience across tasks remains challenging. External memory requires repeated retrieval and interpretation, while trajectory-based training embeds experience in model parameters, tying the incorporation of new experience to further model training. We introduce Knowledge-Augmenting Implicit Memory (KIM), which uses latent-space knowledge augmentation to retain learned behavioral adjustments in a memory separate from the backbone. Building on a fixed shared adapter, KIM learns local adjustments that favor successful continuations and jointly fits a memory field over their associated interaction states. This field integrates adjustments across trajectories to guide behavior in new states. During a new task, KIM reads a state-conditioned adjustment from the field and applies it through a low-rank intervention in the frozen backbone. Tool observations refresh the readout, guiding successive decisions without replaying source trajectories. With a memory field built from 3.6k successful trajectories selected by similarity to evaluation queries, KIM achieves 41.6% average accuracy across seven computational and knowledge-intensive reasoning benchmarks, compared with 30.8% for trajectory RAG using a 54k-trajectory retrieval bank. Applied to Tool-Star, KIM further raises average accuracy from 46.6% to 50.7%, extending its benefits to a model trained for multi-tool reasoning. Our code is available at https://anonymous.4open.science/r/KIM-63C8.