acceptodds
Under review as a conference paper at ICLR 2027

ENGRAM: From Experience to Skills through Verified Consolidation

Abstract

Language-model agents use experience through external retrieval and parameter learning. Consolidating stored experience into model weights requires deciding both which experience to train on and when its external records can leave retrieval. Admission based solely on student mastery can exclude useful teacher demonstrations, while removal immediately after training can discard guidance that the model still needs. We introduce Engram, a framework that separates training admission from memory release. A demonstration-aware gate admits skills based on student mastery or sufficient successful teacher demonstrations. The student learns from selected action traces with context from an Engram Trace Graph (ETG). A verification gate then compares the updated model without episodic retrieval against the base model with retrieval on reserved probes. Release requires valid training participation and scores meeting both relative and absolute criteria; records for skills that fail verification remain retrievable. Experiments across interactive and reasoning benchmarks show consistent gains over frozen-weight ETG controls, with ablations supporting demonstration-aware admission and selective adapter activation. A controlled held-out comparison shows that verified release substantially reduces the retrievable footprint while remaining close to retain-all performance and outperforming immediate and budget-matched random release. Engram also improves task performance while reducing input-token usage and model calls. Skill-level verification further reduces retrievable experience through selective archival, retaining external guidance for skills that do not meet the release criteria.

Primary area:
transfer learning, meta learning, and lifelong learning
Keywords:
LLM agents, memory consolidation, self-evolving agent, knowledge distillation

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_emYHOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper addresses a critical issue in continual learning for agents and systematically investigates the transfer of experiences from external memory to model weights using the ENGRAM framework. The proposed method features a novel design—specifically, decoupling training admission from memory release and introducing a verification gate to prevent the premature deletion of experiences. Experiments span multiple agent benchmarks, and the importance of each module is validated through ablation studies and controlled memory-release experiments.

Critical strengths

Clearly Defines Key Issues in Agent Experience Consolidation

This paper addresses a significant yet under-explored challenge in long-term agent learning: determining when to incorporate external experiences into model parameters and when it is safe to discard external memory. Decoupling training admission from memory release represents a valuable contribution to existing memory-based agent evolution methods.

Proposes a Verification-Driven Memory Release Mechanism

The paper introduces a verification gate that determines whether the model has genuinely assimilated an experience by comparing a model updated without retrieval against a baseline model utilizing retrieval. Compared to strategies that simply delete memory after training, this mechanism adopts a more cautious approach; controlled experiments demonstrate that it reduces the retrieval footprint while maintaining strong task performance.

Critical weaknesses

Reliability of Memory Release Verification Requires Further Validation

This paper uses behavioral consistency on reserved probes as the criterion for successful empirical transfer. However, performance gains on specific probes do not necessarily indicate that the model has acquired a generalizable skill. Particularly in complex interactive tasks, it remains to be investigated whether a limited set of probes can adequately cover the full scope of a skill's capabilities.

Heavy Reliance on Skill Representations and Long-term Evolutionary Processes

ENGRAM assumes the existence of a predefined skill taxonomy and ETG structure; however, issues such as skill discovery, skill granularity, and the construction of inter-skill relationships in real-world open environments have not been fully addressed. Furthermore, the accumulation of adapters for different skills across multiple rounds of consolidation warrants further analysis to determine whether it leads to parameter interference or catastrophic forgetting.

Questions for authors

Q1. The verification gate uses reserved probes to determine whether memory can be released. Given that the number of probes is limited and may not cover the skill's entire behavioral space, how do the authors ensure the reliability of the release decision? Have the authors conducted sensitivity analyses regarding different probe counts or probe selection strategies?

Q2. The skill taxonomy and ETG play a significant role in ENGRAM's performance. Does the method remain stable if the granularity of skill definitions changes?

Q3. During multi-round consolidation, is there a risk of conflict between the LoRA adapters of different skills? Have the authors observed adapter interference or issues with forgetting during long-term training?

Q4. Current experiments rely primarily on existing benchmarks; do the authors plan to evaluate ENGRAM's long-term operational capabilities in more open-ended and dynamically changing environments?

2 comments
AuthorsOpenReview ↗

Reply to reviewer emYH

Thank you for the positive assessment and the precise questions. Here are our reply regarding your questions.

1. Q1: reliability of release from a limited probe set

  • Episode isolation. Probes are whole held-out episodes, reserved before training and never trained on. Their steps are assigned to the skill the deployment router would choose.
  • Minimum evidence. Release requires route-matched probe steps from at least 2 independent episodes, terminal-step accuracy of at least 0.5 and no worse than the memory arm, and an adapter that matches or beats both the plain base model and the retrieval-equipped base. On ALFWorld that is 32 probe steps from 8 episodes per skill; on HotpotQA, 8 questions per skill.
  • ALFWorld, 3 training seeds. Probe verification released 3, 5 and 4 of the 6 skills and kept 19–55% of the records. It scored at least as well as retaining everything: +2.0 [−2.0, +6.0], n.s. (per seed +3.0, 0.0, +3.0). It beat random release of at most the same number of records by +7.8 [+3.9, +11.8], above each of the 15 random draws by 0.7 to 17.9 points. Within a seed, all variants share that seed's adapter, so the probes picked skills whose records were safe to release.
  • HotpotQA. Verification released 1 of 2 skills and scored 0.8–1.0 EM below both retaining and releasing everything (both significant). Here the decision was no better than a fixed policy.
  • WebShop. Two of five page types had no probe episodes, and the other three did not pass verification, so nothing was released.
  • Sensitivity. We did not run a probe-count or probe-selection sensitivity study. The probes are informative where a skill is well covered, but the decision is coarse and can be conservative; we state this as a limitation.

2. Q2: skill granularity

  • Skills are fixed, environment-defined routes: the 6 ALFWorld task types parsed from the goal, the 5 WebShop page types, and HotpotQA question forms. ENGRAM does not discover skills.
  • Sensitivity. Across the 15 ALFWorld random draws the cost ranged from −0.7 to −17.9 points depending on which skills were released. Identical releases differed by up to 3.7 points, so part of that spread is re-evaluation noise, but most is not: the outcome depends on which units are verified. We list granularity as a limitation.

3. Q3: adapter interference and forgetting

  • Design. Each round trains a fresh LoRA candidate from the base model on all admitted skills of that round (no stacking of adapters); deployment maps each skill to the adapter that passed its check.
  • What this does not show. It avoids accumulating adapters but does not rule out interference between skills trained together. The multi-round runs' in-loop evaluation is too small to measure per-skill forgetting, so we make no claim either way and state this as a limitation.

4. Q4: open-ended environments

  • We agree this is the important next step. The current results are limited to benchmarks with fixed task types, and the revision states that scope explicitly.

Hi Reviewer emYH,

We sincerely thank you for reviewing our paper and for helping us improve our work. With only a short time remaining in the discussion period, we would be very grateful to know whether our responses have addressed your concerns. In particular, we are eager to learn whether our latest response has resolved any remaining issues. We would be happy to continue the discussion if anything remains unclear.

Reviewer_JtrHOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

This work proposes a framework to gradually internalize external skills in model weights. The method first chooses which skill to train on next based on the number of available records and demonstrations. It then trains the model on the selected skill, and only removes the skill from the memory if the trained model on its own outperforms the base model with skill retrieval. Otherwise the context is retained with either the trained model or base model, depending on which one performs best. Experiments on sequential decision-making and question answering tasks show that the proposed method outperforms most baselines.

Critical strengths
  • The aim of gradually internalizing external information in the model weights is interesting
Critical weaknesses
  1. I have concerns about novelty of the proposed method. The overall framework is basically about (1) training on a skill once enough demonstrations are available for it (2) checking whether the trained model improves enough compared to when it has access to the skill. Each step is done with simple threshold-based rules and training with SFT, so the novelty seems limited.

  2. The experiments compare the proposed method to a wide range of baselines without discussing why they are appropriate or how they ensure a fair comparison. For example, some of the baselines do not use any external skills (like CoT, GRPO, ReAct, Reflexion, etc), others involve skills but it is unclear how the skills/demonstrations available in the baseline implementations compare to the ones available to the proposed method. The experimental details do not provide any information on which skills are made available to the model, how demonstrations are collected, and how this compares to the data available to other baselines. Given the framework is about how to iteratively internalize skills in model weights and reduce the number of skills stored, I would expect experiments more targeted towards this objective, for example studying the evolution of performance over iterations, and focusing the comparison to the baselines most relevant to this objective.

  3. The writing is hard to follow and the presentation could be improved significantly. Many terms are introduced without being clearly defined (eg the “Retrieve” function l.160, “router” l.324, how the “teacher” is instantiated is never mentioned, etc). It is unclear how the proposed Engram Trace Graph (ETG) compares to context engineering methods from prior works. The description of the method uses vague language (e.g., “Successful traces supply action targets; failure corrections are included only when they provide a valid action aligned with an observation.” l.279). Experimental details about which skills/demonstrations/other external information is available to the method and to the baselines are lacking. Figure 4 has text annotations that seem out of place (“Replay: not instantiated” l.415).

Questions for authors
  1. How do you ensure a fair comparison with the baselines in terms of available external information, iteration budget, etc?
4 comments
AuthorsOpenReview ↗

Reply to reviewer JtrH - Part 1

Thanks for your review. There might be some misunderstand for the paper, below we clarify all the points you raised. Every point in the review is already addressed in the submission. Below we give the location of each.

1. Q1: fairness in available information and iteration budget

The paper makes two different comparisons and says which is which (App. D, "Comparison conditions").

  • Task-level baselines (Table 2). Each method runs as implemented, with its own learning and inference procedure, under the benchmark configurations of App. B; all scores come from our own runs. They compare systems as designed, not under matched information, and the paper does not present them as matched. They cover the mechanisms of Table 1:
    • prompting with no stored experience (CoT, ReAct);
    • self-reflection across trials (Reflexion);
    • weight learning from the environment alone (GRPO);
    • external memory without weight updates (Mem0, MemRL, MAGE);
    • joint weight and memory updates (EvolveR, SkillRL, EDGE), the group closest to ENGRAM's objective.
  • Component ablations (Tables 3–4). Matched traces and task splits: information is held fixed and one component changes. Three rows are exactly the controls you ask for: "ETG with frozen weights" (same experience, retrieval only, no training), "SFT, no inference ETG" (training on the same experience, no retrieval) and "Mastery-only admission" (same experience, without demonstration-aware admission). Table 4 holds the checkpoint fixed and varies only which records are released.
  • Skills and demonstrations available. Skills are the fixed per-benchmark taxonomy; demonstrations come from the teacher through the environment (Sec. 4.1, App. A.2). Initial retrieval records per benchmark are in Table 7.
  • Budget. The scaled experiments use a single batch update followed by per-skill verification (App. A.1). Table 7 gives the SFT budget per benchmark (3 data passes, optimizer updates). Matched-data controls reuse the same acquired experience. Acquisition, verification, training and serving are accounted for separately (App. B.2, App. E); serving cost per method is in Table 5.

2. Weakness 2: experiments targeted at the objective

  • The objective is to remove external records without losing performance. Table 4 tests exactly that: held-out performance after each release policy, on one checkpoint, against retaining everything.
  • Why one round. A single consolidation round (App. A.1) isolates the release decision; over several rounds, training and release effects would mix. Retrieval occupancy over rounds follows Eq. (9).

(Continued in the next comment.)

AuthorsOpenReview ↗

Reply to reviewer JtrH - Part 2

3. Weakness 1: novelty

Simple rules are not a lack of novelty. The contribution is not any single threshold. It is which evidence authorizes which decision, and the result that admission and removal must be decided separately:

  • Training is authorized by demonstrations, not only by mastery. A mastery-only rule never trains a skill the student cannot yet perform.
  • Removal of a skill's records is authorized only by a held-out test: the trained model without retrieval must match the base model with retrieval. Training participation shows that the records were used, not that the model no longer needs them; only the held-out test checks that.
  • Activation: an adapter that fails that test can still be used together with retrieval, per skill.

Positioning (Sec. 2, Table 1). Memory-based agents (Reflexion, Voyager, MAGE, MemRL, Memento-Skills) adapt without updating the acting model's weights. Methods that update both weights and memory (EvolveR, AgentEvolver, SkillRL, EDGE) do not make removal of a stored record conditional on a held-out test of whether the trained model still needs it; EDGE prunes by utility. Verified memory release is the part no prior entry has.

4. Weakness 3: ETG versus context engineering

Context-engineering methods decide what goes into the prompt; the weights never change. ETG links each record to the skill it trains, which makes three things possible:

  • Admission counts that skill's demonstrations.
  • Release removes only records that took part in training the verified adapter, keeping a few recent residual records (App. A.3).
  • Routing deploys adapter, adapter + ETG, or Base+ETG per skill.

A prompt-curation method has no counterpart to the last two, because nothing it stores is ever replaced by weights.

5. Weakness 3: terms the review calls undefined

  • Retrieve (l.160): defined at first use in Sec. 3: c_j = Retrieve(x_j, M_t) is the episodic context from the current store, and c_j = ∅ without retrieval.
  • Router (l.324): Sec. 4.3 and App. F, "Routing implementation". It maps a request or state to the skill taxonomy using only deployment-visible inputs; it never sees gold answers, episode success, future states, verification outcomes or held-out labels. It then loads that skill's latest approved adapter, or Base+ETG.
  • Teacher: Sec. 4.1 and App. A.2 specify how it is instantiated: it acts through the same environment interface as the student, and the environment's outcome decides demonstration success. The model is Qwen3.5-27B.
  • l.279: App. A.2 states the rule precisely: "A correction enters action training only if it supplies a valid action for an aligned observation." A teacher-written correction is trained on only when it contains an executable action paired with the observation it answers; text-only advice is never a training target.
  • Figure 4, "Replay: not instantiated": deliberate. Eq. (7) includes replay, and Fig. 4 is the single-checkpoint study, where replay is empty (App. A.1). The label says exactly that.

Each question in your review is answered by the submission: the comparison design (App. D), the matched controls you ask for (Tables 3–4) and the definitions (Sec. 3–4, App. A, E, F). Based on the above explanation, we sincerely hope you can re-judge the paper, including its soundness and presentation scores. If you have extra questions, we are glad to offer more detailed explanation.

Hi Reviewer JtrH,

Thank you for reviewing our paper and for helping us improve our work. With only a short time remaining in the discussion period, we would be very grateful to know whether our responses have addressed your concerns. In particular, we are eager to learn whether our latest response has resolved any remaining issues. Apparently there was some misunderstand and we sincerely hope you can re-judge the paper and raise the score based on a more clear consideration. We would be happy to continue the discussion if anything remains unclear.

Thank you for your response. My concerns about the novelty and the scope of the experimental validation remains. I would encourage the authors to make the experimental setup more transparent to make clearer what the proposed method aims to improve over comparable baselines, rather than relying on broad comparisons to many baselines where most do not seem directly comparable to the proposed method. I will retain my score.

Reviewer_QX73OpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

The paper proposes ENGRAM. The key idea is to decide which skills / stored memory is no longer needed and can be emitted after a model is trained with trajectories coming from a stronger teacher model.

Critical strengths
  • Good motivation and important research question: The problem of skill selection and memory management is important and of interest to the community
Critical weaknesses

In the paper's current presentation, it's missing many of the necessary details to understand how the experiments are carried out, which makes it very difficult to judge whether the experiments offer a fair comparison between ENGRAM and baselines. In the paper's current form, so many details are missing so that we couldn't understand the full details, let alone reproduce the experiment setting and results. I would strongly suggest the authors spend more effort and work on improving the presentation of the paper.

Examples of missing descriptions in the paper:

  • No information on what the "teacher" demonstration come from : The paper contains "teacher demonstration " is a key part of the Engram Trace Graph for student to learn from, but there's no description whatsoever in the experiment settings where is being used as teacher and where the demonstrations come from.

  • Only hand-wavy description exist on where the training data come from: For the 600 SFT examples used for HotpotQA, the paper mentions "16 original records and 584 added from official SFT targets", without any explanation on what official SFT target means, how the sampling is done, etc.

  • Criteria of student "mastery": e.g. under what condition do student "master" the task so that skills can be emitted. The paper is offering no detail on the exact parameters and settings for this.

  • Ambiguous terminologies and the exact setting: The paper needs to provide precise definition or reference for terms like "student", "teacher", "skills", "mastery", etc.

Questions for authors

See above

3 comments
AuthorsOpenReview ↗

Reply to review QX73 - Part 1

Thanks for your review. There might be some misunderstand for the paper, below we clarify all the points you raised. The setup is specified in the submission. We give the location of each item, and state the teacher model and the source of the 584 records directly.**

1. Fair comparison

  • Two comparison types (App. D, "Comparison conditions"). Table 2 evaluates each baseline as implemented, with its own learning and inference procedure, under the benchmark configurations of App. B. All baseline scores come from our own runs, not from published results.
  • Matched controls. Tables 3–4 hold the experience and task splits fixed and change one component of ENGRAM. Table 4 also fixes the checkpoint (App. E.1).
  • Cost accounting. Student acquisition, teacher acquisition, verification, training and serving are accounted for separately (App. B.2, App. E). Matched-data controls reuse the same acquired experience. Our reply to Reviewer JtrH gives more detail.

2. Teacher demonstrations (Sec. 4.1, App. A.2)

  • Source. The agent requests demonstrations for low-mastery skills. The teacher acts through the same environment interface as the student, and the environment's reported outcome decides whether a demonstration succeeded.
  • Storage. ETG stores each demonstration with its skill, outcome and source tag (demonstration, correction or student trace).
  • Deployment. The teacher only produces demonstrations and corrections during acquisition. At deployment only the student runs: each skill uses its approved adapter or Base+ETG (Sec. 4.3, App. F).
  • Model. The teacher is Qwen3.5-27B. It is never trained; only the student's adapter is updated.

3. "16 original records and 584 added from official SFT targets" (Fig. 4)

  • Source. "Official SFT targets" means the benchmark's own training labels. The 584 records are questions from HotpotQA's official training split, each with its gold answer as the target, drawn deterministically from that split. They were added to the 16 records the agent had collected itself.
  • No leakage. No evaluation question is among them: App. E deduplicates normalized questions across partitions before answers are attached.
  • Use. The 600 records form the store of the release study in Fig. 4 and Table 4, which uses one fixed checkpoint (App. E.1).

4. Mastery, and when records are removed

The question "under what condition do students master the task so that skills can be emitted" misreads the method. Mastery does not decide when records are removed. Keeping these two decisions apart is the paper's first design choice (Sec. 3, "Two decisions per skill").

  • Mastery m_{t,s} ∈ [0, 1] estimates the student's own success on skill s from evaluation (Sec. 3). With sr the critic's success rate on the skill's evaluation trials, m ← λ·sr + (1−λ)·m when sr ≥ m, and m ← m − μ·(m − sr) otherwise. Successful teacher demonstrations do not raise it. It enters only the admission rule, Eq. (2).
  • Admission, Eq. (2). Skill s is trained if (m_{t,s} ≥ τ_m and n_{t,s} ≥ n_min and r_{t,s}) or d_{t,s} ≥ d_min (Sec. 4.1). n counts stored success and failure records; d counts successful teacher-demonstration records, with d_min = 2 (App. A.3); r means the skill has not been consolidated before, or its mastery has risen by a set amount since. The second branch trains a skill even at m = 0.
  • Release, Eq. (6). Decided after training, and only by reserved probes. The adapter without retrieval must score at least δ above the base model with retrieval (a negative δ allows a bounded drop) and strictly above the floor α. Training participation must be valid and the probes non-empty. Mastery does not appear in Eq. (6).
  • Probes (Sec. 4.3, App. E). Successful trajectories reserved before training and kept out of the training batch. On ALFWorld, identical observation–action–prompt trajectories are grouped across skills, and reserved groups are kept across rounds; other sequential environments reserve a fixed tail of successful records per skill. Sequential probes score agreement with the recorded action, given up to 16 history entries. A QA probe question counts as correct when answer F1 ≥ 0.5, and release requires more than 50% correct.
  • Skills that fail Eq. (6) keep their records. They run adapter + ETG if the activation probe approves it, otherwise Base+ETG (Sec. 4.3).
  • Thresholds. τ_m, d_min, δ and α are the configured inputs of Algorithm 1; d_min = 2 and the QA floor are stated in App. A.3 and App. E.

(Continue in next comment)

5. Terminology (each term is defined where it is introduced)

  • Student: the acting model whose LoRA adapter is trained (Qwen3-8B; Llama-3.1-8B-Instruct in App. C).
  • Teacher: the stronger model that produces demonstrations (Sec. 4.1, App. A.2); Qwen3.5-27B.
  • Skill: an entry of the fixed per-benchmark skill taxonomy in ETG (Sec. 3, App. F).
  • Router: maps each request or interaction state to a skill using only deployment-visible inputs (Sec. 4.3, App. F).
  • Retrieve: c_j = Retrieve(x_j, M_t), the episodic context drawn from the current store (Sec. 3).
  • Mastery, admission u, participation b, release z: Sec. 3, Eqs. (2) and (6).

6. Reproducibility

  • Training: App. F, Tables 7–8 give every SFT hyperparameter per benchmark (data passes, optimizer updates, learning rate, LoRA settings, seed).
  • Data and evaluation: App. B and E give the models, data partitions, probe construction and statistics; App. H gives the prompt interfaces.

Every point in your review is answered above, with its location in the submission. Based on the above explanation, we sincerely hope you can re-judge the paper, including its soundness and presentation scores. If you have extra questions, we are glad to offer more detailed explanation.

Hi Reviewer QX73,

Thank you for reviewing our paper and for helping us improve our work. With only a short time remaining in the discussion period, we would be very grateful to know whether our responses have addressed your concerns. In particular, we are eager to learn whether our latest response has resolved any remaining issues. Apparently there was some misunderstand and we sincerely hope you can re-judge the paper and raise the score based on a more clear consideration. We would be happy to continue the discussion if anything remains unclear.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.