acceptodds
Under review as a conference paper at ICLR 2027

BISE: Brain-Inspired Memory Consolidation for Self-Evolving Agents

Abstract

Accumulating usable knowledge through autonomous exploration of an unfamiliar environment is a core capability of self-evolving agents. The challenge is to preserve the context that makes experience reusable while keeping emerging rules open to correction. We present BISE, a knowledge-accumulating agent inspired by the complementary-learning-systems view of biological memory. BISE separates ordered accounts of individual attempts from a compact, source-linked summary, allowing knowledge to be revised without erasing the experience behind it. On six Jericho games with GLM-5.3, BISE achieves a mean normalized score of 55.7%, 22.1 points above its matched ReAct control and 18.0 points above reproduced EvoTest. Gains over ReAct hold on two further backbones. In ScienceWorld, the 211-pair panel reaches a mean cumulative best score of 95.5 across up to five attempts. Ablations favor ordered episode context, while trajectory audits trace revised advice to subsequent actions. Together, these findings support retaining the context behind reusable procedures while keeping the knowledge drawn from them open to correction.

Primary area:
foundation or frontier models, including LLMs
Keywords:
language agents, memory-augmented agents, episodic and semantic memory, memory consolidation, in-context learning, LLM agents, long-horizon decision making, safety and hazard avoidance, text-based interactive environments, brain-inspired AI

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_gPJgOpenReview ↗
Rating
2
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

The paper asks how an agent can reuse experience across repeated exploration without discarding action order, prerequisites, and failure context, or freezing an incorrect interpretation. BISE uses two memory layers: an episode-indexed gist stores ordered experience from one attempt, while a source-linked schema page aggregates and revises reusable rules across attempts. It evaluates six Jericho games and 211 ScienceWorld task variations. On GLM-5.3, the mean score is higher than ReAct and reproduced EvoTest, but the ablations suggest that the main benefit comes from ordered gists. But the schema page’s additional performance contribution remains unclear.

Critical strengths
  1. The problem and solution line up. The paper does not merely ask for “better memory”; it separates preserving ordered experience from revising accumulated knowledge, and assigns those functions to the gist archive and schema page.
  2. The experience unit is concrete and auditable. Episode gists retain action order, prerequisites, outcomes, and failure context. Source links and trajectory audits let the reader trace how advice was formed.
Critical weaknesses
  1. Only one layer of the two-layer solution has stable performance support. The ordered-gist direction is supported by an ablation, but gist-only is nearly as strong as full BISE. The schema page’s added value is mainly shown through organization and traceability, not a stable task-performance gain.
  2. The ScienceWorld memory result lacks a key control, and the release materials are incomplete. Without a repeated-attempt no-memory system, the improvement cannot be separated from gains caused by the task or the actor’s repeated attempts. The selected cases also do not estimate the rate of wrong advice, overgeneralization, or false rules. The paper says that supplementary material contains scripts and snapshots, but gives no anonymous source link for independently checking the implementation and audit process.
Questions for authors
  1. Can you add a same-backbone, same-step-budget, same-attempt-count repeated-attempt no-memory control on ScienceWorld and report paired results for each task variation?
  2. Can you repeat the de-temporalized gist, verbatim log, gist-only, and full-BISE comparisons across multiple seeds, with paired confidence intervals or at least run-level distributions?
  3. Can you quantify incorrect or unverified rules in the schema page and gists, wrong retrieval advice, and their consequences, while reporting token, curator-call, and inference costs rather than only environment interaction budgets? Please provide accessible anonymous links to the source code, run scripts, and audit data during review.
2 comments
AuthorsOpenReview ↗

Response to Reviewer gPJg [Part 1]

W1. What does the schema page add?

Full BISE and gist-only achieve similar scores across three runs (55.7±2.1 and 55.0±2.7), while BISE records 7.6% fewer tokens per attempt: 1.73±0.08M versus 1.88±0.09M, including actor and curator usage. The final knowledge files contain 4,685 words, compared with 17,281 in the episode archive, summed over six games and averaged across runs. The page provides a compact view; the archive is still retained.

The Library trace shows how this view is revised. After attempt 4 succeeds with the original book, the curator replaces the page's incorrect copying rule with the technician–printer–librarian procedure. Attempt 5 retrieves the updated page and repeats that procedure. This directly documents the page's role in updating advice while preserving the earlier episode record.

W2 / Q1. ScienceWorld beyond retrying

On the same 63 initially imperfect task–variation pairs, the complete memory procedure reaches 89.5±1.8 versus 83.0±2.2 for memoryless retries over four attempts and three runs per arm, a 6.6-point difference. Mean solved counts are 50.7 versus 39.0. Both arms allow 100 actor rounds and 100 environment steps per attempt; the memory arm also allows 15 curator rounds. The selection attempt contributes neither scores nor memories to these restarted runs.

Mean cumulative-best scores across attempts are 67.5 → 81.4 → 86.3 → 89.5 with memory and 71.5 → 80.2 → 81.8 → 83.0 without. Task-aligned differences between arm means are positive for 29 pairs, zero for 25, and negative for 9. Runs are the replication unit because variations can share memory within a run; the task differences are descriptive.

Scores use environment records for the requested task and variation. Missing or wrong-task records add no verified progress, earlier verified best scores carry forward, and all 63 pairs remain in the denominator. We retain the recorded early stops, including stops without a verified solve. Both arms configure the same service alias; the archive does not pin the served revision. The comparison evaluates the complete procedures under a common interaction budget. No-memory run 1 has four attempts; the five-attempt comparison uses runs 2 and 3 in both arms and gives 91.3±1.0 versus 82.9±2.1. The original 211-pair result is a separate within-system evaluation.

Q2. Repeated representation comparisons

All five representations now have three independent sequential runs, each containing six games and ten memory-sharing attempts per game, capped at 110 steps per attempt. Individual run scores and mean ± s.d. are:

  • Verbatim event log: 49.6, 53.2, 50.8 (51.2±1.9).
  • Atomized triples: 49.7, 42.9, 40.1 (44.2±4.9).
  • De-temporalized gist: 48.2, 54.6, 52.8 (51.9±3.3).
  • Gist-only: 52.0, 57.2, 55.9 (55.0±2.7).
  • Full BISE: 57.9, 55.3, 53.9 (55.7±2.1).

Statistics use unrounded scores. We report run distributions because the records do not identify matched seeds across conditions. Atomization has the largest mean loss, 11.5 points. The de-temporalized condition changes the summary instructions and retrieval-score presentation as well as order: content and length are not fixed. Its 3.8-point gap compares that configuration with BISE rather than isolating event order.

AuthorsOpenReview ↗

Response to Reviewer gPJg [Part 2]

Q3. Incorrect advice and consequences

We have not estimated population-level rates of incorrect rules in pages or gists, wrong retrievals, or harmful advice. The available action-level evidence traces a corrected rule and an unsupported generalization to their subsequent use.

In Library run 1, attempt 4 executes ask technician about gates (+2), examine printers (+5), and give librarian printout (+2), then leaves with the original book. The curator replaces the copying rule before attempt 5 retrieves the updated page and repeats the successful procedure. In Zork 3, the page records contrasting teleport outcomes but also overgeneralizes that every non-“I” indicator phase is fatal. Attempts 7–10 reach 6/7 without recorded deaths; those executions do not test the universal claim. The records therefore show both a correction and a rule that remains unvalidated.

A separate source-traceability screen covers 463 entries in 18 final primary pages. Of these, 435 have source-attempt markers and 28 do not. Among the marked entries, 371 meet a 40% content-word overlap threshold with the cited gists and 64 fall below it. These are lexical source-support categories, not correctness labels; auxiliary files and gist correctness are outside this screen.

Q3. Token use and curator calls

BISE records 1.73±0.08M tokens per attempt, comprising 1.60M actor and 0.13M curator tokens, with 6.7 usage-bearing curator calls. Curation accounts for 7.6% of the total. Gist-only records 1.88±0.09M tokens and 6.9 curator calls. The ReAct actor records 1.22M tokens, making BISE's recorded total about 42% higher than this actor-only control.

Counts include resumed runs and are normalized by 60 planned attempts per run. Failure logs without usage are excluded from token sums, so these are recorded inference costs rather than complete billing totals. ReAct's unused curation has incomplete logs and is excluded. Comparable EvoTest usage, monetary cost, and wall-clock latency are unavailable.

The supplementary material includes anonymized per-response token records and run-level summaries.

Reviewer_k5RMOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The authors propose BISE, a continual learning framework (termed self-evolving by the authors) inspired by hippocampal indexing theory and complementary learning systems. The framework consists of two interacting components: a fast component responsible for action selection and a slow component responsible for consolidating acquired knowledge. The proposed approach generally appears to outperform the evaluated baselines. However, the main manuscript does not clearly establish whether these improvements are consistent and substantial across the evaluated experiments.

Critical strengths

The authors attempt to connect their proposed framework to hippocampal indexing theory and complementary learning systems, two influential theoretical frameworks in neuroscience. Drawing inspiration from these theories may provide a potentially interesting biological perspective on continual learning.

Critical weaknesses
  1. Overall, the manuscript suffers from substantial issues with clarity, organization, and poor writing quality, which makes it difficult to assess the proposed method and its contributions. For some reasons, the writing seems to be heavily AI generated. Also, the research problem formulation is not clearly established. Despite reading the manuscript multiple times, it remains unclear whether the authors are addressing a reinforcement learning problem or a more general continual learning setting. Given the references to actions and rewards in the experimental setup, I shall assume the former, but this should not be the case.

  2. Despite drawing inspiration from hippocampus indexing theory and complementary learning systems, the main manuscript does not explain how these theories motivate the specific algorithmic choices beyond the related work section. It will be greatly appreciated if the authors can make a clearer connection between the underlying theoretical principles and the proposed architecture. This would be helpful to better understand the biological motivation of their proposed approach.

  3. Several figures (such as Figure 1) appear to have been generated or heavily stylized using AI tools. Regardless of how they were produced, their presentation could be improved to communicate the proposed architecture and experimental findings more clearly. For example, more explanations within the figure captions are needed. They should explain more clearly and with more depth, what the results suggest or insights gained from the results.

  4. Several important concepts and components (such as gist, page, and index) of the proposed framework are introduced without adequate definitions or explanations, making it difficult to follow the technical development. I will leave some comments on these in the Questions section.

  5. The manuscript does not clearly explain what each experiment is intended to evaluate. I recommend explicitly stating the research questions or hypotheses associated with each environment and benchmark, and explain how the experiments test these concerns or unknowns.

  6. There may be interesting ideas and empirical findings in this work. But, the current writing and presentation obscure these potential contributions, making it difficult to determine the extent of the scientific contribution. In my opinion, a substantial re-writing effort is required for this paper to be ready for publication at a top-tier ML conference.

  7. Experimental results are evaluated using only three random seeds, which can be insufficient to establish the reliability of the results. I recommend evaluating using at least five seeds, with additional runs where necessary depending on the observed variability.

Questions for authors
  1. What does BISE stand for? Is it just a random made up name?

  2. What does gist and advice refer to ?

  3. Why does Figure 3 not show the results of the baseline models? On that note, what are the baseline models? What did BISE bring that they do not?

  4. What does n in Table 2 refer to? Why is it not consistent?

2 comments
AuthorsOpenReview ↗

Response to Reviewer k5RM [Part 1]

W1 / W6. Problem setting

Our problem setting follows the repeated-episode test-time learning setup studied in EvoTest (He et al., 2026). An agent repeatedly attempts the same environment, using experience from earlier attempts to improve subsequent performance. In BISE, the environment resets between attempts, while external text memory persists and the LLM weights remain frozen. Rewards evaluate outcomes and are recorded with the experience; adaptation occurs through memory updates. Our specific question is what structure this memory should preserve to support procedure reuse and revision. EvoTest evolves the broader agent configuration, whereas BISE keeps the update procedure fixed and adapts the stored episode accounts and schema page.

W2 / W3. Biological motivation and Figure 1

The actor selects actions within an attempt; the curator updates memory after the attempt ends. Both roles use the same frozen model. The fast/slow distinction therefore concerns when the operations occur, rather than separately trained networks.

Hippocampal indexing motivates linking an episode account to its source attempt. Complementary learning systems motivate separating individual experience records from knowledge integrated across experiences. BISE implements these ideas by appending an ordered gist, revising a cross-attempt schema page, and retrieving from either store on later attempts. Updating a page leaves the earlier gist available. These are functional design analogies.

For example, after a Library attempt succeeds with the original book, the curator replaces the page's copying requirement with the successful procedure; the next attempt retrieves the revised page. We will update Figure 1 in the final manuscript to label the shared model, the within-attempt and between-attempt operations, and the retrieval paths explicitly, and make its caption explain this update cycle.

W4 / Q1 / Q2. Names and definitions

BISE means Brain-Inspired memory for Self-Evolving agents. A gist records one attempt's ordered route, conditions, outcomes, and failures. A schema page summarizes procedures, hazards, and open questions across attempts and is revised when new evidence arrives. Advice is text retrieved from either store for the actor to use, adapt, or ignore. Episode-indexed means linked to the source attempt; a separate embedding index supports similarity search. These definitions now appear when the objects are introduced.

AuthorsOpenReview ↗

Response to Reviewer k5RM [Part 2]

W5 / Q3. What the experiments test and what the baselines retain

Jericho tests whether the complete procedure improves repeated task solving over memoryless and adaptive agents. The representation comparisons test which information to retain; gist-only tests the incremental role of the page. ScienceWorld compares the memory procedure with repeated attempts without memory. The trajectory cases examine how a saved revision enters later retrieval and execution. A separate GLM-5.3 run extends each game to 50 attempts to examine longer histories: the mean is 64.8 overall, 55.1 over the first ten attempts, and 68.4 over the last ten.

For the baseline missing from the original Figure 3, the ScienceWorld comparison now gives 89.5±1.8 versus 83.0±2.2 on the same 63 pairs over four attempts and three runs per arm. Both arms allow 100 actor rounds and 100 environment steps per attempt; the memory arm adds 15 curator rounds. The cumulative-best scores use verified environment records and retain all pairs and recorded early stops. This comparison evaluates the complete memory procedure against memoryless retries.

The Jericho baselines retain different information. ReAct carries no cross-attempt memory; RecMem consolidates experience into facts. The EvoTest scaffold provides Naive (no memory), Sliding window (recent states), Summary (a short progress summary), and RAG (retrieved within-attempt interactions). EvoTest additionally adapts prompts, extraction code, and memory between attempts. BISE stores a complete ordered episode account alongside a revisable source-linked summary.

Our EvoTest actor and evolver both use GLM-5.3, as do BISE's actor and curator; the original EvoTest uses Gemini-2.5-Flash or Claude-4-Sonnet actors with an o3 evolver. With ten 110-step attempts per game, the three-run Jericho means are 55.7±2.1 for BISE versus 37.7±2.6 for reproduced EvoTest, an 18.0-point gap. BISE exceeds matched ReAct by 22.1 points. All ± values are run standard deviations.

W7 / Q4. Repeated runs and n

n counts independent sequential runs, each containing ten memory-sharing attempts per game on six games. All five memory representations now have n=3. BISE's scores are 57.9, 55.3, and 53.9; gist-only's are 52.0, 57.2, and 55.9. The other means are 51.2±1.9 for verbatim logs, 44.2±4.9 for atomized triples, and 51.9±3.3 for de-temporalized gists. The no-memory row has n=1, while program pre-extraction has n=2 and is reported separately.

Each independent run builds its own memory through agent–environment interaction. We retain per-attempt environment trajectories, saved memory snapshots, and run-level scores to trace the reported results to the underlying executions. The run scores and standard deviations above summarize variability across these independent repetitions. An anonymized experiment index and structured score and token records are included in the supplementary material. We will release the implementation and experiment configurations upon acceptance.

Reviewer_Da8qOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

BISE stores experience in two forms: an append-only archive of ordered episode gists and a compact schema page that can be revised while retaining links to the supporting attempts. A curator reads each completed trajectory, updates these stores, and makes the resulting entries available to the actor through retrieval. The paper evaluates repeated attempts on six Jericho games and a ScienceWorld task panel. On the main GLM-5.3 evaluation, BISE reports a mean normalized score of 55.7%, compared with 33.6% for matched ReAct and 37.7% for reproduced EvoTest. The ablations suggest that ordered gists retain most of the benefit, while the schema page mainly provides an inspectable organization of the accumulated knowledge.

Critical strengths
  1. The main Jericho comparison shows a substantial gain under a shared backbone and environment-interaction budget. BISE and reproduced EvoTest each have three independent sequential runs, with means of 55.7 and 37.7 and run standard deviations of 2.1 and 2.6, respectively (Table 1). The leave-one-game-out analysis keeps the margin positive, and the matched ReAct comparisons extend to two other backbones. This is useful evidence that the complete memory procedure improves repeated task solving within the evaluated setting.

  2. The paper makes its memory revisions unusually inspectable. The Library example traces an incorrect copying rule to a later correction and subsequent successful executions; the Zork 3 example preserves preparation that earned no immediate reward (Figure 5 and Appendix B.1). The authors also state what these records do not establish: generated explanations can be wrong, text changes are not accuracy measurements, and selected cases are not causal estimates. This careful separation makes the architectural contribution easier to assess.

Critical weaknesses
  1. The central evidence about memory representation is still under-replicated. In Table 2, the verbatim, atomized, and de-temporalized variants each have only one sequential run. The 7.5-point drop after removing order is suggestive, but its variability is unknown. The paper also does not specify the de-temporalization procedure well enough to determine whether prerequisites, content length, and retrieval behavior remain comparable. Meanwhile, gist-only scores 55.0 ± 2.7 versus 55.7 ± 2.1 for the full system, so the current experiment gives no clear performance benefit for the second memory layer. The authors acknowledge this. Its incremental empirical value is therefore limited to the documented organization and revision examples; the strongest performance evidence concerns the ordered gist archive.

  2. The ScienceWorld result cannot yet establish that the additional progress is caused by memory. The headline 95.5 is a cumulative-best score after up to five attempts, and 148 of 211 pairs already reach 100 on their first attempt. Selecting the remaining 63 initially imperfect pairs and measuring improvement on another attempt leaves both retry effects and selection effects unresolved without a matched repeated-attempt control. The paper correctly acknowledges the missing control, but this limits how much the second benchmark adds to the evidence for the proposed memory procedure.

Questions for authors
  1. How exactly is the de-temporalized gist constructed? Please clarify which text, prerequisites, retrieval settings, and length constraints are preserved, and provide independent-run results for this variant if they already exist. This determines whether the observed gap can be attributed to event order.

  2. Does the existing ScienceWorld evaluation include a no-memory run using the same task pairs, stopping rule, and five-attempt budget? If so, please provide its cumulative-best and per-attempt results. If not, please clarify whether the current results support a causal memory benefit beyond the benefit of trying again.

1 comment
AuthorsOpenReview ↗

Response to Reviewer Da8q

W1 / Q1. Replication and the de-temporalization procedure

The de-temporalized condition now has three independent runs, scoring 48.2, 54.6, and 52.8: 51.9±3.3, compared with 55.7±2.1 for BISE. The mean gap is 3.8 points, replacing the original single-run estimate of 7.5. The other three-run means are 51.2±1.9 for verbatim logs, 44.2±4.9 for atomized triples, and 55.0±2.7 for gist-only. All ± values are run standard deviations; attempts within a run share memory.

The condition rewrites both the knowledge page and episode record as unordered bullets, prohibits numbered steps and route narration, and omits the standard 800-word page target. It does not preserve the same text, prerequisites, or length. All runs retain ten attempts, 110 actor rounds, and 30 curator rounds. The archived retrieval code keeps similarity ranking and filtering, then displays similarity as 0.5; it does not shuffle entries or remove attempt labels. This display is recorded in runs 1 and 3; equivalent return records are unavailable for run 2.

For a concrete example, the Detective record retains take paper in the Chief's office with its +10 reward as a location–action–reward bullet. The same output still says to visit other rooms “before any risky confrontation” and describes a hazard occurring “after roughly six turns.” Thus some temporal and conditional relations remain despite the unordered format.

The 3.8-point difference measures this joint configuration, not event order alone. We now describe the ablation accordingly and frame the representation result as support for retaining episode-level context.

W2 / Q2. ScienceWorld control, stopping, and per-attempt results

The existing records provide a common four-attempt comparison across three runs and a five-attempt comparison for runs 2 and 3 in both arms; no-memory run 1 has four attempts. The comparison restarts on the same 63 initially imperfect pairs, without carrying over the selection attempt's scores or memories. Each attempt permits 100 actor rounds and 100 environment steps; the memory arm adds 15 curator rounds.

At four attempts, mean cumulative-best scores are 89.5±1.8 with memory versus 83.0±2.2 without, with 50.7 versus 39.0 solved pairs. At five attempts, runs 2 and 3 give 91.3±1.0 versus 82.9±2.1. For those same two runs, cumulative-best scores across attempts 1–5 are 69.2, 81.1, 86.5, 89.6, 91.3 with memory and 70.8, 79.9, 80.9, 82.1, 82.9 without.

Scores within each individual attempt, rather than cumulative best, are 70.3, 65.7, 64.1, 60.5, 47.8 with memory and 73.1, 61.6, 53.5, 53.6, 48.7 without. These average valid environment trajectories within each run and then average the two run means. Valid trajectory counts across the two runs are 124, 67, 38, 27, 18 and 122, 69, 54, 53, 47, respectively. Completed tasks leave the active set, so these per-attempt scores describe changing subsets; the cumulative-best comparison retains all 63 tasks per run.

Cumulative best uses scores from environment actions on the requested task and variation. Wrong-task or missing records add no verified progress, and earlier verified best scores carry forward. Recorded stopping decisions are retained, including early stops triggered by summary success without a verified solve. Both arms configure the same service alias; the archive does not pin the served revision. The result compares the complete memory procedure with memoryless retries under the recorded execution protocol. The original 211-pair score of 95.5 remains a separate within-system result.

The supplementary material includes anonymized environment score records and the per-pair run endpoints.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.