acceptodds
Under review as a conference paper at ICLR 2027

Use and Disuse: Intent-Structured Experience Consolidation for Memory and Learning in LLM Agents

Abstract

The evolution of Large Language Model agents from single-task execution to long-term autonomous operation highlights the critical challenge of transforming continuous experiences into reusable knowledge. To address this, we propose Hippocam, a hierarchical memory and continual learning architecture. Hippocam draws inspiration from two characteristics of human memory: cognitive processes selectively maintain information relevant to current goals, while long-term memories form gradually through repeated consolidation. Accordingly, Hippocam structures an agent’s ongoing work as nested intents. The active context remains centered on the current intent, while completed intents are consolidated into the task-relevant outcomes and state needed for subsequent work, rather than carrying forward their full working details. Concurrently, a recursive prefix consolidation mechanism repeatedly consolidates earlier history, causing long-unused experiences to become increasingly abstract. Original interactions are preserved, allowing the agent to progressively recover finer-grained details through the hierarchy and stop once sufficient information is available. Crucially, when past experiences are recalled and reintegrated into active work, they undergo subsequent consolidation alongside new experiences, thereby being reinforced, supplemented, and updated. Through this memory dynamic of use and disuse, Hippocam connects working context, long-term memory, knowledge accumulation, and skill learning within a single continuously evolving experiential process. This enables agents to learn and evolve capabilities through their own experiences without parameter updates.

Primary area:
transfer learning, meta learning, and lifelong learning
Keywords:
LLM Agents, Agent Memory, Continual Learning, Experience Consolidation, Hierarchical Memory, Intent-Structured Memory, Long-Term Memory

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_DayiOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

The paper introduces Hippocam, a hierarchical memory and continual-learning architecture for LLM agents. It organizes ongoing work into nested intents, compresses completed intents into task-relevant outcomes, recursively consolidates older prefixes into increasingly abstract representations, and preserves the underlying interaction history in a tree that can be progressively recalled when finer detail is needed. The paper validates Hippocam across learning, transfer, and long-horizon memory benchmarks. On ALFWorld, ScienceWorld, and StreamBench it reports 97.8%, 81.9%, and 81.7%, respectively, with Hippocam best or tied-best on two of the three.

Critical strengths
  • This work creates a natural cycle in which experience is retained, reused, updated, or allowed to recede over time. This architecture is well suited to the main challenges facing long-running LLM agents today: limited context capacity, interference from irrelevant past information, loss of important details during summarization, and difficulty turning past experience into useful knowledge for future tasks.
Critical weaknesses
  • The proposed use and disuse mechanism is supported more strongly as an intuitive explanation than as a demonstrated causal mechanism. The transition analysis shows that knowledge associated with intervening use is more frequently supplemented or updated, which is encouraging, but use is measured through indirect proxies and the authors correctly note that the observed association does not establish causality. More importantly, the quantitative analysis does not establish the corresponding disuse effect: most items with no detected use remain unchanged between adjacent consolidations, while evidence for gradual fading comes mainly from qualitative traces.
Questions for authors

N.A.

2 comments

Thank you for your thoughtful review and constructive feedback. We appreciate your recognition of Hippocam's potential for long-running agents. We hope our response below addresses your concerns.


Weakness: Empirical Evidence for Use and Disuse

Response: We clarified that use and disuse are emergent behaviors of Hippocam's architecture (Section 2.6), and added empirical evidence for both.

  • For use, we manually traced knowledge supplementation and updating to identify whether each change arose from use or observations (Table 16). 74.4% of knowledge supplementation and updating involved changes derived from use, showing that most updates come from use.

    Table 16. Sources of knowledge supplementation and updating. Paired values denote Supplemented / Updated; Use (%) includes Use-only and Mixed events.

    BenchmarkTotalUse-onlyMixedObs.-onlyUse (%)
    ALFWorld204 / 41101 / 1530 / 773 / 1964.2 / 53.7
    ScienceWorld27 / 1916 / 41 / 610 / 963.0 / 52.6
    ToolBench188 / 29169 / 262 / 117 / 291.0 / 93.1
    Overall419 / 89286 / 4533 / 14100 / 3076.1 / 66.3
    All5083314713074.4
  • For disuse, we clarified that disuse emerges as consolidation abstracts and prunes prior experience. Although PALIM preserves useful knowledge, items may still be shortened or discarded, especially under token budget constraints (Algorithm 2). Most knowledge remained unchanged in Appendix A.6 because ToolBench ran only six consolidations without reaching the token budget. We added an analysis of 23 ALFWorld consolidations (Figure 7), where knowledge items were shortened 151 times and discarded 40 times, with 41 items shortened as early as the 11th consolidation without reaching the token budget.

Thank you for your thoughtful review and valuable feedback. With the discussion deadline approaching, we would greatly appreciate hearing if you have any remaining concerns following our response and additional analyses. We would be happy to address them promptly.

Reviewer_324hOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes Hippocam, an LLM agent that manages long-term interaction history through nested intents, intent-conditioned consolidation, recursive prefix consolidation, and progressive recall. Its main contribution is to connect working context, long-term memory, and experience updating in one continuously evolving process, evaluated on ALFWorld, ScienceWorld, StreamBench, LoCoMo, and MemoryAgentBench.

Critical strengths
  1. The paper presents a meaningful unified architecture. It does not simply add a separate memory store; instead, task state, intent boundaries, historical compression, and recovery of original evidence are handled as one continuous context-management process.
  2. The evaluation covers experience accumulation, cross-domain transfer, long-context memory, conflicting information, and component ablations.
Critical weaknesses

The experiments support Hippocam’s overall effectiveness, but the interpretation of the “use and disuse” mechanism goes somewhat beyond the evidence: the results mainly show narrative abstraction and knowledge revision following related work, rather than systematic fading of unused knowledge. Most evaluations also use single runs and imperfectly matched baseline resources and protocols.

Questions for authors
  1. Could the authors provide the main baseline comparisons under matched context, tool, and reasoning budgets, or at least provide a complete resource-accounting table?
  2. In the current PALIM prompt, previously extracted knowledge is instructed to preserve identifying details, while the narrative is what becomes progressively coarser. Does “disuse” refer to narrative abstraction, deletion of knowledge content, or future reorganization of knowledge?
  3. Can the authors provide results across multiple random seeds, or at least paired variance and confidence intervals under fixed task orders?
  4. Can the gains from continual learning be separated from task order, accumulated feedback, and changes in task difficulty? Do you have controls with cleared memory, frozen memory, or identical later-task sequences?
3 comments

Thank you for your careful review of our paper and valuable suggestions. We are encouraged by your positive comments on our architecture and evaluation. We hope the following responses adequately address your concerns.


Q1: Resource-Aligned Comparisons

R1: We added a resource checklist (Table 6). Context window sizes are design-specific hyperparameters, and each method's default settings can help achieve strong performance, so we followed these settings to ensure a fair evaluation. Meanwhile, we aligned auxiliary resources by enabling reasoning and removing the output limits for mem0 and A-Mem's auxiliary LLM calls, and Hippocam still leads. For the remaining differences, we ran matched-budget checks (Table 7). Even with main-agent reasoning turned off to match Letta, Hippocam outperforms Letta on interleaved GoodAI tasks.

Table 6. Resource configuration. Unset windows default to 1M; peaks denote the largest observed context or output.

MethodContext windowReasoning (main)Reasoning (auxiliary)Auxiliary output limit
Hippocam128kOnOnNone
HiAgent8,192OnOnNone (peak 130)
Letta128kOff (forced)OnNone
mem0Unset (peak 56.7k)OnOnNone
A-MemUnset (peak 102.4k)OnOnNone
ReasoningBankUnset (peak 16.3k)OnOnNone
ACEUnset (peak 32.5k)OnOn4,096
LTMAgent16kOnOnNone

Q2: Definition of Disuse

R2: Disuse covers all three aspects mentioned. We clarified that disuse is an emergent behavior as consolidation abstracts and prunes prior experience (Section 2.6). This includes details pruned by Intent Consolidation, narrative abstraction, and knowledge shortening, deletion, and reorganization.

  • Although PALIM preserves useful knowledge, items may still be shortened or discarded, especially under token budget constraints (Algorithm 2). Disuse affects narratives and knowledge at different rates, as shown by our new analysis of 23 ALFWorld consolidations (Appendix A.6).

Q3: Statistical Uncertainty and Robustness

R3: Thank you for this constructive suggestion.

  • We added 95% paired confidence intervals for the performance difference between Hippocam and each baseline across all main experiments (Table 5). The intervals are entirely positive for Hippocam in 103 of the 137 comparisons, including gains over the strongest baselines of +38.3 pp [22.3, 55.3] on interleaved GoodAI, +27.4 pp [21.0, 33.7] on MAB, and +6.5 pp [4.6, 8.3] on LoCoMo.

  • We reran ScienceWorld, which has the widest confidence intervals and may be sensitive to order, under two additional task orders (Table 10). Hippocam achieves the highest mean progress and the lowest standard deviation.

    Table 10. ScienceWorld mean target progress (%) across three task orders.

    MethodOriginalOrder 1Order 2Mean ± SD
    Hippocam81.985.387.584.9 ± 2.8
    ACE (4,096)91.975.382.583.2 ± 8.3
    ACE (no limit)91.973.267.577.5 ± 12.8
    A-Mem89.175.874.379.7 ± 8.1
    mem081.265.785.777.5 ± 10.5

Q4: Controls for Continual Learning

R4: Our original description of RQ2 was too brief. We clarified in Section 4.2 that our experimental setup controls for all three factors. First, all conditions evaluate the same target tasks to keep difficulty identical. Second, Frozen resets to the source snapshot before every task, ruling out feedback accumulation and task-order effects. Third, Fresh (cleared memory) and After run the same task sequences.

  • On both ALFWorld-to-ScienceWorld and HotpotQA-to-FEVER transitions, Hippocam Frozen outperforms Fresh (78.9% vs. 55.9% and 61.8% vs. 53.9%).

  • Table 10 also evaluates sensitivity to task order, and we clarified this in Section 4.1.

Thank you for the detailed response. The additional experiments and analyses increase my confidence in the reliability of the empirical results. However, some limitations remain regarding the evidence for the core mechanism, so I will keep my original scores.

Thank you for taking the time to review our response. We appreciate your careful consideration of our work.

Reviewer_aY5aOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes Hippocam, a hierarchical memory architecture that enables LLM agents to continually learn from accumulated experience without updating model parameters. The framework organizes ongoing work into nested intents using JANUS, consolidates completed tasks into reusable knowledge through PRECIP, and recursively compresses older experiences using PALIM while preserving original interactions for progressive recall. Through this use-and-disuse mechanism, relevant knowledge can be reinforced or revised through subsequent experience, while less frequently used experiences become increasingly abstract. The authors evaluate Hippocam across benchmarks covering continual learning, cross-domain transfer, and long-term memory reliability, comparing it against six existing methods. The results show strong performance across diverse settings, leading or matching the best baseline on seven of eight reported measures. Additional ablation studies, case analyses, and computational cost evaluations further investigate the contributions and effectiveness of the proposed mechanisms.

Critical strengths
  • The paper integrates intent-aware context management, recursive experience consolidation, and progressive recall into a coherent framework. By preserving original interactions while continuously refining reusable knowledge, Hippocam addresses the practical challenge of balancing information compression, long-term retention, and learning from experience.

  • The paper evaluates continual learning, cross-domain transfer, and long-term memory reliability across diverse benchmarks, demonstrating strong performance against existing methods. Additional ablation studies and case analyses provide supporting evidence for the benefits of intent-conditioned consolidation, recursive compression, and progressive recall.

Critical weaknesses
  • The experiments do not fully control for inference computation, reasoning settings, context budgets, and tool access across methods. In particular, some baselines have reasoning disabled, while Hippocam uses additional reasoning and tools. These differences make it difficult to determine how much of the reported performance advantage comes from the proposed memory architecture.

  • Although the framework integrates several memory management components, the experiments do not fully establish their advantages over simpler alternatives. For example, a flat-summary baseline achieves the same 100% success rate on ALFWorld, while the analysis provides limited evidence that unused knowledge progressively loses detail. This weakens the empirical support for the claimed use-and-disuse mechanism and its necessity.

Questions for authors
  1. The appendix acknowledges differences in reasoning settings, computational budgets, and tool access across methods. Could the authors clarify which additional tools Hippocam uses in StreamBench and whether its main performance advantages would hold under comparable reasoning, tool access, and inference budgets?

  2. In Table 9, both Hippocam and the flat-summary baseline achieve 100% success on ALFWorld, although Hippocam requires fewer actions. What evidence most clearly demonstrates the additional benefit of intent-structured and recursive consolidation over simpler summarization? A comparison under matched resource budgets or on tasks without ceiling-level performance would help clarify this.

  3. The paper emphasizes that unused knowledge gradually loses detail through the use-and-disuse mechanism. However, the PALIM instructions explicitly preserve previously extracted knowledge, and Appendix A.6 shows that most knowledge items without detected use remain unchanged. Could the authors clarify whether disuse primarily affects episodic narratives or reusable knowledge itself, and what experimental evidence supports the claimed behavior?

4 comments

Thank you for your careful review of our paper and thoughtful comments. We are encouraged by your positive comments on our framework design, experimental performance, and ablation studies. We hope the following responses adequately address your concerns.


Q1: Experimental Resources and Fairness

R1: Hippocam's additional tools in StreamBench are bash, grep, find, glob, read, and write, which are unrelated to the tasks. We removed them and re-ran the evaluation, obtaining identical results, and updated the description accordingly.

  • Context window sizes are design-specific hyperparameters, and each method's default settings can help achieve strong performance. We therefore followed these settings to ensure a fair evaluation (Table 6).

    Table 6. Resource configuration. Unset windows default to 1M; peaks denote the largest observed context or output.

    MethodContext windowReasoning (main)Reasoning (auxiliary)Auxiliary output limit
    Hippocam128kOnOnNone
    HiAgent8,192OnOnNone (peak 130)
    Letta128kOff (forced)OnNone
    mem0Unset (peak 56.7k)OnOnNone
    A-MemUnset (peak 102.4k)OnOnNone
    ReasoningBankUnset (peak 16.3k)OnOnNone
    ACEUnset (peak 32.5k)OnOn4,096
    LTMAgent16kOnOnNone
  • To ensure fairness, we aligned resources for affected baselines. We enabled reasoning and removed the output limits for mem0 and A-Mem's auxiliary LLM calls, and Hippocam still leads. Even with main-agent reasoning turned off to match Letta, Hippocam outperforms Letta on interleaved GoodAI tasks (Table 7).

    Table 7. Matched-budget checks. Default and Matched denote the original and aligned settings; Reference is the method being matched.

    SettingDefaultMatchedReference
    HiAgent window, 8,192 → 128k (ref. Hippocam)
    ToolBench (150 tasks)61.369.376.0
    MAB 32k / 64k0.0 / 11.014.0 / 0.077.0 / 41.0
    ACE auxiliary output limit, 4,096 → none (ref. Hippocam)
    ScienceWorld91.991.981.9
    FEVER Fresh / After / Frozen61.8 / 63.8 / 57.958.0 / 62.0 / 58.053.9 / 59.9 / 61.8
    Hippocam main-agent reasoning, on → off (ref. Letta)
    ToolBench (150 / 300 tasks)76.0 / 78.773.3 / 76.076.7 / 77.3
    FEVER Frozen61.860.059.9
    GoodAI isolated / interleaved85.1 / 88.774.4 / 60.986.7 / 50.4

Q2: Comparison with Flat-Summary

R2: Thank you very much for your constructive suggestions.

  • Our ablation study's original goal was to evaluate the necessity of keeping knowledge as is, so we replaced only Hippocam's Prefix Consolidation output with a flat summary and further clarified the variants in Appendix A.4. We agree that ALFWorld is relatively simple. Although both Hippocam and the flat-summary variant reach 100% success, the flat-summary variant requires 34 recalls, while Hippocam requires none.

  • Following your suggestion, we added evaluations on harder ScienceWorld tasks (Table 13). Hippocam achieves 88.1% mean progress, outperforming the flat-summary variant at 55.3%.

    Table 13 (excerpt). Prefix consolidation and alternative knowledge configurations on ScienceWorld.

    ConfigurationProgress (%)Mean stepsRecalls
    Hippocam88.122.80
    Empty placeholder65.330.50
    ACE playbook58.032.10
    Flat summary55.330.11
  • Furthermore, the ablation study in Section 5.1 demonstrates the benefit of intent-structured consolidation, as removing it retains more irrelevant information and significantly reduces performance.


Q3: Disuse of Knowledge

R3: Although PALIM preserves useful knowledge, items may still be shortened or discarded, especially under token budget constraints (Algorithm 2). We clarified that disuse is an emergent behavior as consolidation abstracts and prunes prior experience (Section 2.6).

  • Disuse affects narratives and knowledge at different rates. Most knowledge remained unchanged in Appendix A.6 because ToolBench ran only six consolidations without reaching the token budget.

  • We added an analysis of 23 ALFWorld consolidations (Figure 7), showing that 41 items were shortened as early as the 11th consolidation. When the budget was reached at the 21st consolidation, 37 items were discarded and 67 were merged or split.

  • The intent-conditioning ablation in Section 5.1 also supports disuse during Intent Consolidation.

Thank you for your time and effort in reviewing our paper. As the discussion deadline is approaching, we would like to ask whether you have any remaining questions or concerns regarding our response. We would be happy to provide further clarification promptly.

Thank you for the detailed rebuttal and additional experiments. The new ScienceWorld comparison substantially addresses my concern about the benefits over simpler summarization. I also appreciate the resource-aligned evaluations and the additional use-and-disuse analysis. However, the resource comparisons still do not fully control for total inference budgets, and the reasoning-aligned results with Letta are mixed. The new analysis demonstrates progressive abstraction and knowledge compression, but does not clearly isolate the role of non-use from repeated consolidation and token-budget constraints. These limitations remain relevant to my assessment, so I will maintain my original rating.

Thank you for carefully considering our response and for your thoughtful feedback. We appreciate your recognition of our additional experiments and analyses, as well as your pointing out the mixed results with Letta.

To further address this concern, we adapted Letta to support reasoning while preserving its tool-based execution. Specifically, we relaxed the mandatory tool-call requirement to allow reasoning. If the model did not invoke a tool, we prompted it to do so, retrying up to three times. We reran all evaluated tasks with this adaptation and verified that reasoning remained enabled throughout.

Table. Performance comparison (%) with Letta's original and reasoning-enabled configurations. Slash-separated values follow the order specified in each task.

TaskLetta (Original)Letta (Reasoning On)Hippocam
ALFWorld85.186.097.8
ScienceWorld (RQ1)71.263.981.9
StreamBench DDX (150 / 300)86.0 / 91.783.3 / 88.087.3 / 93.0
StreamBench Tool (150 / 300)76.7 / 77.376.7 / 78.376.0 / 78.7
StreamBench Mean (150 / 300)81.3 / 84.580.0 / 83.281.7 / 85.8
LoCoMo overall51.769.877.6
MAB (6k / 32k / 64k)76.0 / 36.0 / 20.090.0 / 57.0 / 40.097.0 / 77.0 / 41.0
GoodAI (isolated / interleaved)86.7 / 50.466.7 / 45.085.1 / 88.7
ALF→Sci (frozen / fresh / after / Δ)40.8 / 59.3 / 54.5 / −4.844.8 / 61.3 / 60.5 / −0.878.9 / 55.9 / 75.1 / +19.2
FEVER (frozen / fresh / after / Δ)59.9 / 69.8 / 59.9 / −9.963.9 / 71.8 / 65.8 / −6.061.8 / 53.9 / 59.9 / +6.0

Hippocam still outperforms reasoning-enabled Letta on most evaluations. Reasoning substantially improves Letta on LoCoMo and MAB, while reducing its performance on GoodAI. We observed that Letta's context exceeded the 1M-token window and was truncated, which may explain the substantial performance drop on GoodAI.

Regarding disuse, we would like to clarify that disuse is not a separately designed mechanism, but an emergent behavior of Hippocam's architecture, describing the evolution of experience as a whole. The knowledge is one component. At the knowledge level, the observed changes arise through recursive consolidation. When further reduction is required under the token budget, PALIM is simply prompted to produce a more concise output, without any additional processing of either narrative or knowledge. Thus, the role of non-use is closely intertwined with recursive consolidation and token-budget constraints.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.