acceptodds
Under review as a conference paper at ICLR 2027

Principles that Guide, Actions that Inform: Agent Evolution via Knowledge Abstraction

Abstract

Large language model (LLM) agents have demonstrated strong capabilities in interactive environments, yet their ability to continually evolve from experience remains limited. Although fine-tuning provides a pathway for adaptation, its reliance on parameter access and substantial computational resources limits its flexibility and scope of application, particularly for large-scale and closed-source LLMs. External memory provides an alternative by enabling agents to accumulate experience without modifying model parameters. However, existing approaches primarily focus on how experiences are represented and organized, while the acquired knowledge often remains closely coupled with specific tasks and contexts, limiting its generalization for continual agent evolution. A fundamental challenge is therefore how to transform concrete interactions into abstract and reusable knowledge that can guide future decisions and enable agents to generalize beyond individual experiences. To address this issue, we propose SAGA (Self-evolving Agents through Experience-Grounded Abstraction), a framework for experience-grounded knowledge abstraction and utilization in LLM agents. SAGA progressively transforms interaction trajectories into episodic descriptions, reusable procedures, and principles with explicit applicability conditions, while preserving links to their supporting execution evidence. To make abstract knowledge actionable, SAGA instantiates retrieved principles into task-specific operational guidance and uses principle-derived checks to refine candidate actions through corrective feedback and resampling. These mechanisms form an execution–abstraction feedback loop, in which accumulated knowledge guides subsequent interactions and newly collected experiences update the hierarchical memory. Experiments on ScienceWorld and ALFWorld demonstrate improved task performance, while ablation studies show that contextual instantiation and action regulation are important for realizing the benefits of principle-level knowledge.

Primary area:
foundation or frontier models, including LLMs
Keywords:
LLM agents, knowledge abstraction, continual agent evolution, memory system

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_ZxvaOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes SAGA, a memory framework for LLM agents. Trajectories are abstracted into four levels: raw trajectories, episodic summaries, procedures, and principles with applicability conditions. At test time, retrieved principles are rewritten into task-specific constraints, and some principles are also used to check candidate actions and trigger resampling. The method is evaluated on ScienceWorld and ALFWorld, both with frozen API models and with SFT + RL on Qwen2.5-7B.

Critical strengths

The observation that injecting abstract principles directly can hurt, while instantiating them for the current task helps, is interesting and useful for work on agent memory beyond this paper.

The memory construction is careful, and the appendix is honest about several confounds and limitations.

Critical weaknesses
  1. The evidence does not support the main claims. In the ablation, most of the gain comes from simply retrieving raw trajectories; the parts that are new in this paper add only a few points on top of that, from single runs without variance estimates. The generalization claim is also not supported: on ALFWorld the strong model already solves the unseen task types without memory, and the 7B improvement cited in the main text actually comes from the trajectory-only control, as the appendix acknowledges.

  2. The gains cannot be clearly attributed to the method. The training-based results use trajectories from much stronger models for SFT, with no control trained on the same data without the knowledge context, so they may largely reflect distillation. Baselines are copied from other papers, while the authors' own harness gives a noticeably different score for the same base model. The closest prior work (CLIN, AutoGuide, AutoManual) is discussed but not compared.

Questions for authors
  1. Can you report results over multiple seeds, or with confidence intervals, for the main and ablation tables?
  2. How was the reported round or checkpoint chosen? Please also report last-round results.
  3. Can you add SFT on the same trajectories without knowledge context, and a setting where the same model builds and uses the memory?
4 comments
AuthorsOpenReview ↗

Response to Reviewer Zxva (Part I)

We thank the reviewer for the detailed assessment and for identifying the central attribution and evaluation questions. We address the weaknesses (W) and questions (Q) in order below.

W1(a) & Q1: Gains beyond trajectory retrieval and statistical reliability

Raw trajectory retrieval provides a substantial initial improvement. The key question is whether SAGA adds a reliable gain beyond this strong control. We therefore repeated the complete ScienceWorld ablation using five independently constructed memories. Within each memory, all configurations use the same test instances, actor, source experiences, context budget, and evaluation harness.

ConfigurationMean ± SD
R2: L0 trajectory retrieval63.7 ± 0.6
R3: L1 + L264.5 ± 0.7
R4: L1 + L2 + direct L363.9 ± 0.8
R5: L1 + L2 + instantiated L366.2 ± 0.7
R6: R5 + action regulation67.0 ± 0.7
R7: Outcome-validated SAGA72.4 ± 1.1

The submitted full configuration improves over trajectory retrieval by 3.3 points, with a positive improvement in all five independently constructed memories. Outcome-validated SAGA further increases the improvement to 8.7 points (95% CI: [7.5, 9.9], paired p = 0.004). It also exceeds the submitted full configuration by 5.4 points (95% CI: [4.3, 6.5], paired p = 0.006).

These results clarify the roles of the components. Raw trajectories provide strong task-specific guidance. Hierarchical abstraction, task-conditioned instantiation, and action regulation add a consistent gain, while outcome validation produces the largest additional improvement by using later execution outcomes to reinforce, narrow, or exclude principles. Revised Table 7 reports mean ± SD for every ablation row and replaces the original single-run interpretation with the five-memory statistical analysis.

W1(b): Generalization with a non-saturated actor

The Qwen3.8-Max ALFWorld setting is saturated: the no-memory actor already obtains 100% on both Look and Pick2. SAGA retains 100% on Look and obtains 94.1% on Pick2 because one game reaches the context limit. The transfer analysis therefore uses the fixed Qwen2.5-7B actor, whose no-memory performance remains far from saturation.

The 57.5% result in the submission corresponds to trajectory retrieval, while the submitted complete hierarchy reaches 59.4%. Under the same fixed actor, held-out task types, interaction budget, and evaluation harness, outcome-validated SAGA reaches 70.1%. Look and Pick2 remain excluded from experience collection and principle validation, and the memory is frozen before evaluation.

ConfigurationLookPick2Overall
No memory18.936.545.5
Trajectory-only50.047.157.5
Revised SAGA77.870.670.1

Compared with trajectory retrieval, revised SAGA improves Look by 27.8 points, Pick2 by 23.5 points, and overall performance by 12.6 points. Revised Tables 4 and 5 correct the attribution of the 57.5% result, add the complete 7B comparison, and ground the transfer conclusion in the non-saturated Qwen2.5-7B setting.

AuthorsOpenReview ↗

Response to Reviewer Zxva (Part II)

W2(a) & Q3: Separating distillation, knowledge conditioning, and builder strength

The training-based results contain a substantial teacher-action distillation component. On ScienceWorld, the SFT targets are reconstructed from Qwen3.8-Max evolution trajectories using the score-dependent trajectory-prefix rules specified in the Appendix. To separate teacher-action distillation from hierarchical knowledge conditioning, all SFT variants use the same selected teacher trajectories and action targets. They also share the Qwen2.5-7B actor, GRPO task pool, reward, rollout budget, and evaluation setting. GRPO starts from each SFT checkpoint and uses new on-policy rollouts rather than replaying the teacher trajectories.

Qwen2.5-7B configurationTeacher actionsKnowledge builderSFTSFT + GRPO
Base modelNoNone9.6—
Same teacher actions, no knowledge contextYesNone53.862.7
Same-model SAGA knowledgeYesQwen2.5-7B55.163.8
Full SAGA knowledgeYesQwen3.8-Max58.166.9

The increase from 9.6 to 53.8 shows the large contribution of teacher-action distillation. With teacher actions fixed, knowledge constructed by Qwen2.5-7B further improves SFT by 1.3 points and the final policy by 1.1 points. Replacing the same-model builder with Qwen3.8-Max adds another 3.0 points after SFT and 3.1 points after GRPO. In total, full hierarchical knowledge conditioning improves the matched no-knowledge control by 4.3 points after SFT and 4.2 points after GRPO.

These controls separate three sources of improvement: teacher-action distillation provides the main initial gain, hierarchical knowledge conditioning adds a further gain under fixed supervision, and the stronger builder improves the quality of that knowledge. The positive same-model control also shows that the knowledge pipeline remains useful without a stronger external builder. Revised Table 2 reports the complete attribution study.

W2(b): Same-harness comparison with prior methods

We thank the reviewer for identifying the ambiguity in the submitted comparison. The submission placed results copied from prior papers beside results produced by our harness, although their prompts, model wrappers, action parsers, backbones, and evaluation protocols differed. The revised manuscript separates reported cross-paper results from matched reimplementations.

Within each benchmark, all matched methods use the same actor, source experiences, task split, interaction budget, context budget, and evaluation harness. ScienceWorld uses Qwen3.8-Max. ALFWorld uses Qwen2.5-7B as the actor and Qwen3.8-Max for memory or rule construction. For AutoGuide, we follow its original success–failure trajectory pairing, context identification, guideline extraction, and top-2 guideline selection. Look and Pick2 remain excluded from the shared experience pool.

ScienceWorld methodAvg. Score
No memory48.4 ± 0.5
Trajectory retrieval63.7 ± 0.6
ExpeL65.2 ± 0.9
Memento66.4 ± 1.0
ReasoningBank66.7 ± 0.8
CLIN66.9 ± 0.9
Submitted SAGA67.0 ± 0.7
Revised SAGA72.4 ± 1.1
ALFWorld methodOverall success rate
No memory45.5
Trajectory-only57.5
AutoGuide61.9
AutoManual64.1
Revised SAGA70.1

Submitted SAGA performs comparably to CLIN under the matched ScienceWorld setting. Revised SAGA exceeds CLIN by 5.5 points (95% CI: [4.1, 6.9], p = 0.002). On ALFWorld, revised SAGA exceeds AutoGuide by 8.2 points and AutoManual by 6.0 points. Revised Tables 3 and 6 report these matched comparisons, and the Appendix specifies the corresponding protocols.

AuthorsOpenReview ↗

Response to Reviewer Zxva (Part III)

Q2: Checkpoint selection and last-round performance

The reported 66.9% result was selected without using test-set performance. During GRPO, checkpoints were ranked only by their average reward on the training task pool. The checkpoint with the highest training reward was fixed before test evaluation and then evaluated once on the held-out test set. No separate validation split was introduced.

CheckpointSelection ruleTest performance
Training-selected checkpointHighest average reward on the training task pool66.9%
Final checkpointLast GRPO checkpoint66.2%

The final checkpoint remains within 0.7 points of the training-selected checkpoint. Figure 4 and the accompanying analysis now replace the ambiguous label “Best” with “Training-selected checkpoint,” state the selection rule, and report the final-checkpoint result.

Revision. Changes made specifically in response to Reviewer Zxva are highlighted in purple (#7B1FA2) in the revised manuscript. Revisions shared across multiple reviews are highlighted in orange (#B45309). The revision adds the five-memory ablation and statistical analysis in Table 7; the non-saturated held-out transfer study in Table 5; the matched distillation, knowledge-conditioning, and builder-strength controls in Table 2; the matched prior-method comparisons in Tables 3 and 6; and the checkpoint-selection rule and final-checkpoint result in Figure 4 and the accompanying text.

AuthorsOpenReview ↗

Follow-up on our response

Dear Reviewer Zxva,

Thank you again for your detailed assessment. We have addressed the concerns regarding variance and statistical reliability, non-saturated generalization, the attribution of distillation and knowledge conditioning, matched baseline comparisons, and checkpoint selection, and incorporated the corresponding results into the revised manuscript.

As the discussion period draws to a close, we would appreciate it if you could let us know whether our response and revisions have adequately addressed your concerns. If any issue remains unclear or unresolved, we would be grateful for the opportunity to provide further clarification.

Sincerely,
The Authors of Submission 814

Reviewer_8dRTOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper divides the agent's interaction experience into four levels of knowledge (raw experience, episode knowledge, procedure knowledge, and experience-grounded principles) and constructs a pipeline for extracting these four levels of knowledge hierarchically. The paper proposes SAGA, an evolution framework based on a task execution-knowledge extraction feedback loop. Finally, the framework's effectiveness is validated through comparisons with both prompt-based and training-based baselines on ScienceWorld and ALFWorld.

Critical strengths
  1. The paper has a complete structure and content.
  2. Experiments conducted on two benchmarks against two types of baselines show the effectiveness of SAGA.
Critical weaknesses
  1. Although the L3 principle-level abstraction is emphasized in the paper as one of the core contributions of SAGA, existing experiments have not adequately isolated its independent contribution. In ablation experiments, direct injection of the L3 principle actually led to a performance decrease, while the improvement only appeared after adding contextual instantiation and action regulation. Therefore, it remains unclear whether the performance improvement comes from the principle-level abstraction itself or from the overall process of further reconstructing the retrieved knowledge into task-related guidance.
  2. Although the paper has compared SAGA with several recent agent evolution and memory-based baselines, the existing comparisons mainly focus on raw trajectory retrieval or end-to-end evolution frameworks, and lack a baseline that is more closely matched in terms of experience structuring, knowledge retrieval. In particular, it is currently unclear whether the performance improvement comes primarily from hierarchical principle abstraction or from the additional organization and reasoning processes introduced in the SAGA process.
Questions for authors
  1. Does the L3 principle really bring additional benefits? Could similar improvements be achieved by directly applying task-conditional instantiation to L1 and L2?

  2. In training-based experiments, is some of the improvement attributable to a stronger knowledge construction model Qwen3.8-Max?

4 comments
AuthorsOpenReview ↗

Response to Reviewer 8dRT

We thank the reviewer for the careful assessment and for focusing on the independent contribution of principle-level knowledge and the role of the knowledge builder. We address the weaknesses (W) and questions (Q) in order below.

W1, W2 & Q1: Contribution of L3 beyond additional processing

To isolate the contribution of L3 knowledge from applicability filtering and task-conditioned rewriting, we conducted four matched controls. All controls use the same actor, source experiences, test instances, context budget, and evaluation harness. They are evaluated in the supplementary Qwen2.5-7B utilization setting and exclude predicate-based action regulation.

ConfigurationOperationPaired difference
Direct L3 injectionInject retrieved L3 principles without filtering or rewriting−1.55
L3 filter-onlyRemove NOT APPLICABLE principles without rewriting+0.84
Instantiated L2Apply the same filtering and task-conditioned rewriting to L2+1.27
Instantiated L3Apply filtering and task-conditioned rewriting to L3+3.65

Paired differences are measured against the corresponding no-injection control. Instantiated L3 exceeds filter-only by 2.81 points (95% CI: [1.70, 3.92], p < 0.001), so filtering explains only part of the improvement. It also exceeds instantiated L2 by 2.38 points (t = 3.50). Because the L2 and L3 conditions use the same applicability check and rewriting procedure, this comparison controls for the additional LLM operation. The results locate the contribution in task-grounded L3 knowledge rather than raw L3 text or an additional rewriting step.

We also conducted matched comparisons with closely related experience-based methods. Within each benchmark, all methods use the same actor, source experiences, task split, interaction budget, context budget, and evaluation harness.

ScienceWorld methodAvg. Score
No memory48.4 ± 0.5
Trajectory retrieval63.7 ± 0.6
ExpeL65.2 ± 0.9
Memento66.4 ± 1.0
ReasoningBank66.7 ± 0.8
CLIN66.9 ± 0.9
Submitted SAGA67.0 ± 0.7
Revised SAGA72.4 ± 1.1
ALFWorld methodOverall success rate
No memory45.5
Trajectory-only57.5
AutoGuide61.9
AutoManual64.1
Revised SAGA70.1

Revised SAGA exceeds the strongest matched baseline by 5.5 points on ScienceWorld and 6.0 points on ALFWorld. These comparisons separate the contribution of SAGA from differences in the backbone or evaluation protocol. Revised Tables 3 and 6 report the matched results, while Appendix Table 15 reports the controlled L2/L3 instantiation analysis.

Q2: Contribution of the stronger knowledge builder

Qwen3.8-Max contributes to the training-based result as an offline knowledge builder, while Qwen2.5-7B serves as the execution policy. We measured this contribution by fixing the teacher action trajectories, SFT targets, actor, GRPO task pool, reward, rollout budget, and evaluation setting, and varying only the hierarchical knowledge context and its builder.

Knowledge used by Qwen2.5-7BKnowledge builderSFTSFT + GRPO
No knowledge contextNone53.862.7
Same-model SAGA knowledgeQwen2.5-7B55.163.8
Full SAGA knowledgeQwen3.8-Max58.166.9

The same-model builder improves SFT from 53.8 to 55.1 and the final policy from 62.7 to 63.8. Thus, SAGA remains beneficial when Qwen2.5-7B both constructs and uses the knowledge. Replacing the same-model builder with Qwen3.8-Max provides a further gain of 3.0 points after SFT and 3.1 points after GRPO. Overall, full hierarchical knowledge conditioning improves the matched no-knowledge configuration by 4.3 points after SFT and 4.2 points after GRPO.

Revised Table 2 reports the no-knowledge, same-model-builder, and stronger-builder controls under matched teacher actions and optimization settings. Section 4.1 and the Appendix now identify the actor and knowledge builder separately and specify the SFT data provenance and GRPO rollout procedure.

Revision. Changes made specifically in response to Reviewer 8dRT are highlighted in teal (#00796B) in the revised manuscript. Revisions shared across multiple reviews are highlighted in orange (#B45309). The revision adds the controlled L2/L3 instantiation analysis in Appendix Table 15, the matched experience-based comparisons in Tables 3 and 6, and the no-knowledge, same-model-builder, and stronger-builder attribution study in Table 2.

AuthorsOpenReview ↗

Follow-up on our response

Dear Reviewer 8dRT,

Thank you again for your careful review. We have added controlled experiments to isolate the contribution of instantiated L3 knowledge and to quantify the effects of the same-model and stronger knowledge builders. We have also incorporated these results and clarifications into the revised manuscript.

As the discussion period draws to a close, we would appreciate it if you could let us know whether our response and revisions have adequately addressed your concerns. If any issue remains unclear or unresolved, we would be grateful for the opportunity to provide further clarification.

Sincerely,
The Authors of Submission 814

Thank you for the detailed response and the additional analyses. The response alleviates my concerns through matched L2/L3 instantiation controls, closer experience-based baselines, and explicit attribution of the stronger knowledge builder. My remaining concern is that the controlled L2/L3 comparison is demonstrated in the supplementary Qwen2.5-7B setting, whereas the revised system’s substantial improvement in the main experiments comes from outcome validation. A comparison between L2 and L3 under the same outcome validation procedure would further clarify the contribution of principle-level abstraction in the final system.

AuthorsOpenReview ↗

Response to the Follow-up Concern

Dear Reviewer 8dRT,

Thank you for this precise follow-up. We agree that our previous analyses answered two related but separate questions: the supplementary control compared L2 and L3 under the same instantiation procedure, while the main ablation measured the effect of outcome validation on the full L3-based system. Your suggested comparison is therefore important for determining whether L3 retains an independent contribution in the final system.

We have now conducted this matched comparison in the main ScienceWorld setting that produces the 72.4 result. This is a training-free comparison using the fixed Qwen3.8-Max actor; no model parameters are updated. Both configurations use the same five independently constructed memories, source experiences, retrieval and instantiation budgets, validation-task variants, matched-rollout budget, knowledge-utilization pipeline, action-regulation setting, and evaluation instances. The only difference is the abstraction level subjected to outcome validation. In the L2 configuration, instantiated procedures undergo the same rollout-based utility evaluation and retention criterion used for L3 principles.

ConfigurationAvg. Score
L1 + L2, without outcome validation (reference)64.5 ± 0.7
Outcome-validated L268.5 ± 1.0
Outcome-validated L372.4 ± 1.1

The direct matched comparison is between the latter two rows. Under the same outcome-validation procedure, L3 exceeds L2 by 3.9 points. Thus, applying outcome validation to procedural knowledge already produces a strong result, but it does not account for the full improvement of the final system. Principle-level abstraction provides a further gain when the actor, memories, validation procedure, utilization pipeline, and evaluation instances are held fixed.

This result also clarifies how the two components contribute. Outcome validation identifies which retrieved knowledge remains useful through subsequent execution. L3 additionally represents transferable conditions, effects, mechanisms, scope, and exceptions across related procedures, allowing the validated knowledge to guide tasks beyond a specific procedural pattern. The advantage is consistent in direction with the 2.38-point L3 improvement in the supplementary instantiation control and now also holds under the main backbone and the final outcome-validation procedure.

We thank the reviewer for identifying this missing comparison. We hope this additional result resolves the remaining concern by showing that outcome validation benefits both abstraction levels, while L3 retains a substantial independent contribution in the final system. We would be happy to clarify any remaining questions.

Sincerely,
The Authors of Submission 814

Reviewer_G7KDOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper addresses continual self-improvement of large language model (LLM) agents without parameter updates. It frames the core problem as turning concrete interaction experience into abstract, reusable knowledge that can guide future decisions. The authors propose SAGA (Self-evolving Agents through Experience-Grounded Abstraction), which organizes experience into a four-level memory hierarchy:

  • L0, raw experience: complete interaction trajectories (observations, actions and environment feedback) collected during task execution.
  • L1, episodic knowledge: an LLM-generated summary of each trajectory (goal, key actions, outcome and obstacles), in one-to-one correspondence with L0.
  • L2, procedural knowledge: reusable solution strategies induced from groups of episodes that share a task pattern.
  • L3, experience-grounded principles: rules and constraints inferred from several related procedures and their supporting evidence, expressed as condition-effect-mechanism descriptions.

At decision time, SAGA retrieves knowledge from all levels, using the task description together with the LLM-generated plan. It instantiates the retrieved principles into task-specific guidance. It also compiles them into predicates that reject candidate actions which violate them, triggering resampling. New trajectories are abstracted back into the hierarchy, which closes an execution abstraction loop.

The method is evaluated on ScienceWorld and ALFWorld in two settings:

  • a prompt-based setting with a frozen backbone;
  • a training-based setting, applying supervised fine-tuning (SFT) followed by Group Relative Policy Optimization (GRPO) to Qwen2.5-7B-Instruct.
Critical strengths
  • tries to address a relevant important problem of transferring knowledge from interactions without updating parameters
  • The design choices make the system easy to inspect
  • Table 3 adds one component at a time on two backbones. The negative result that injecting principles lowers performance is reported openly.
  • Principle based checks trigger a resampling rather than replacing the policy entirely, this seems reasonable as a way to enforce learnt constraints.
Critical weaknesses
  1. Gains over the trajectory retrieval are small, and there is no measure of variance (or sign of repeated experimentation)

Tab 3 shows more of the improvements come from L0 retrieval, everything that SAGA adds appears to be a minor improvement. Without variance or standard deviation numbers its hard to understand if the contribution through SAGA is consistent. The claim that "the consistent ordering across both backbones supports" each component cannot rest on sub-point differences from what appear to be single runs.

  1. Principle level knowledge seems to hurt.

For both models R4 < R3. It appears the fix in the paper is through the Instantiate function under §3.3. The text doesn't really explain what this function is doing. what is the prompt, what does it output, how often does it decline to apply a principle, and what is the result of R3 plus an instantiation-style rewrite of L2 only?

  1. ALFworld generalisation claim is contradicted by table 2

Qwen 3.8 max without memory is better than with SAGA. The entirely overall gain therefore seems to come from the seen task types and the benchmark is near saturation for this backbone. The claim that "SAGA maintains strong performance on unseen tasks.." is therefore not really supported.

  1. The alfworld numbers in the paragraph do not show up in the tables. Unless I missed it, I do not see the 57.5% number appear in any of the tables.

  2. Unspecified backbones for baselines. it is unclear if the baselines use the same backbones as SAGA. If not, that makes it an unfair comparison.

  3. Missing some baselines. Table 1 contains none of the experience-based baselines on ScienceWorld. The closest prior works are mentioned but do not feature in the tables without any explanation provided for that exclusion.

  4. It is unclear where the SFT data comes from. If the SFT data comes from stronger models that is akin to distillation and perhaps not comparable to RL.

  5. Inconsistent values of reported numbers and unclear checkpoint selection. Table 1 lists Qwen2.5-7B at 13.9, while the text reports 9.6 measured by the authors. Fig. 4 is labelled "Best: 66.9%" and refers to a “selected checkpoint” without stating the selection criterion or whether a validation split exists.

  6. Missing conceptual grounding for the hierarchy. Doesn't cite the origins for procedural distinction comes from cognitive psychology (Tulving, 1972; Squire, 2004). It was adopted for LLM agents in works like Cognitive Architectures for Language Agents (CoALA; Sumers et al., 2023) and MindForge (Lica et al. 2025). Further, there is no criterion for what makes it to an L3 principle rather than L2. It seems to be defined only by a prompt but the criterion for validity is unclear. Principles may restate also pretrained knowledge; the condition-effect-mechanism schema implies causal claims without any causal identification. The principles may simply restate what the backbone already knows; Fig. 3's subtractive colour-mixing example is a case in point.

Questions for authors
  1. Please report results over at least three seeds, and ideally over independently constructed memories, with confidence intervals. Please include a paired significance test of the full SAGA configuration (R6) against raw trajectory retrieval (R2) in Table 3.

  2. What does Instantiate(...) do? Can the authors rule out that the gain from R4 to R5 comes from filtering out irrelevant principles rather than from grounding them in the task?

  3. On alfworld, Qwen3.8 Max without memory scores 100 on both Look and Pick2, while SAGA scores 100 and 94.1. How do the authors reconcile this with the generalisation claim? Can the authors evaluate on a benchmark or split where the no-memory baseline is not saturated?

  4. Which configuration produces the 45.5% →\rightarrow 57.5% result for Qwen2.5-7B cited in Sec. 4.2, given that it appears in no table? Which backbone does the "Base Model" row in Table 2 use, and why is there no 7B SAGA row in the prompt-based setting?

  5. Could the authors evaluate ExpeL, ReasoningBank and Memento on ScienceWorld, and CLIN and AutoManual on their respective benchmarks, all with the same backbone as SAGA? Or justify their exclusion?

  6. About predicates:

  • How are natural-language principles turned into executable predicates?
  • How often does each predicate fire, and what is its false-positive rate?
  • How many resamples are allowed, and what happens if every resample still violates a predicate?
  • What makes something a predicate vs principle, what's the conceptual grounding here?
  1. Is the four-level hierarchy intended to be cognitively motivated? If so, please relate it to the episodic/procedural memory literature. If not, what justifies the number of levels and the condition-effect-mechanism schema? How does SAGA differ from the evidence based reflections in Generative Agents (Park et al., 2023)?
9 comments
AuthorsOpenReview ↗

Response to Reviewer G7KD (Part I)

We appreciate the reviewer’s detailed assessment and accurate summary of both the training-free and training-based settings. Before addressing the individual weaknesses (W) and questions (Q), we first clarify the central contribution of SAGA.

SAGA’s central contribution is an evidence-linked execution–abstraction loop rather than any single memory level. Concrete interactions support the abstraction of reusable knowledge; the abstracted knowledge is grounded in the current task to guide subsequent actions; and new outcomes refine the utility, scope, exceptions, and status of the acquired knowledge. The training-free setting realizes this loop through hierarchical memory, task-conditioned instantiation, and principle-based action regulation. The training-based setting further uses the accumulated trajectories and knowledge for SFT and GRPO. This process reflects how humans derive general lessons from experience and revise them through practice. To our knowledge, SAGA is the first agent-evolution framework to integrate this knowledge-abstraction-driven loop across both training-free and training-based adaptation.

W1 & Q1: Magnitude and statistical reliability of the gains beyond trajectory retrieval

Following the reviewer’s suggestion, we repeated the ScienceWorld experiments using five independently constructed memories. Within each memory, all configurations were evaluated on the same test instances. The benchmark, test split, actor, source experiences, evaluation harness, context budget, and inference budget remained unchanged.

ConfigurationMean ± SD
R2: L0 trajectory retrieval63.7 ± 0.6
R6: Submitted full SAGA67.0 ± 0.7
R7: Outcome-validated SAGA72.4 ± 1.1

The submitted full configuration improves over trajectory retrieval by 3.3 points, with a positive improvement for all five memories. Outcome-validated SAGA further increases the score to 72.4 ± 1.1, improving over trajectory retrieval by 8.7 points (95% CI: [7.5, 9.9], paired p = 0.004) and over the submitted full configuration by 5.4 points (95% CI: [4.3, 6.5], paired p = 0.006). The improvement over trajectory retrieval is positive for every independently constructed memory.

The additional gain comes from incorporating downstream action outcomes into principle evolution. Candidate principles retain links to their supporting evidence and are applied only when relevant to the current task. Matched rollouts on training or validation variants update their utility, scope, exceptions, and active status. Effective principles are reinforced, context-dependent ones are narrowed, and repeatedly misleading ones are excluded. This turns subsequent execution outcomes into direct supervision for deciding which abstractions should continue to guide future behavior.

Revised Table 7 reports mean ± SD over the five independently constructed memories. Its accompanying analysis reports the paired improvements, confidence intervals, significance tests, and the statistical protocol used to calculate them.

AuthorsOpenReview ↗

Response to Reviewer G7KD (Part II)

W2 & Q2: Contribution of L3 and the role of Instantiate

The R4 < R3 result identifies an important failure mode: abstract principles require task conditioning before they can reliably guide action. We added four controls to separate applicability filtering, task-conditioned rewriting, and abstraction level:

  • Direct L3 injection inserts every retrieved L3 principle verbatim, without applicability filtering or rewriting.
  • L3 filter-only removes principles marked NOT APPLICABLE and inserts the remaining principles verbatim.
  • Instantiated L2 applies the same applicability check and task-conditioned rewriting to L2 procedures, controlling for the additional LLM operation.
  • Instantiated L3 checks whether each L3 principle applies and rewrites every applicable principle into one operational constraint grounded in the current task, plan, and scene.

These controls use the same supplementary Qwen2.5-7B utilization setting and exclude predicate-based action regulation. Paired differences are measured against the corresponding no-injection control.

ConfigurationPaired difference
Direct L3 injection−1.55 points
L3 filter-only+0.84 points
Instantiated L2+1.27 points
Instantiated L3+3.65 points

Instantiate marks 32.4% of retrieved L3 principles as NOT APPLICABLE, preventing irrelevant principles from entering the actor context. Instantiated L3 still exceeds filter-only by 2.81 points (95% CI: [1.70, 3.92], p < 0.001), showing that task-conditioned rewriting provides a substantial benefit beyond filtering. It also exceeds instantiated L2 by 2.38 points (t = 3.50). Because these two configurations use the same applicability check and rewriting procedure, their difference isolates the additional information provided by the L3 abstraction level.

Revised Section 3.3 defines the input, output, and applicability behavior of Instantiate. Appendix Table 15 reports the four controlled comparisons and their statistical analysis.

W3 & Q3: Generalization under a non-saturated backbone

The Qwen3.8-Max comparison is limited by a ceiling effect: the no-memory model already solves all Look and Pick2 games. SAGA retains 100% on Look and obtains 94.1% on Pick2 because one game exhausts the context budget. The generalization analysis is therefore based on a fixed Qwen2.5-7B actor on the same ALFWorld split. This actor is not saturated; Look and Pick2 remain strictly held out from experience collection and principle validation, and the memory is frozen before evaluation.

Qwen2.5-7B, training-freeLookPick2Overall
No memory18.936.545.5
Trajectory-only50.047.157.5
Revised SAGA77.870.670.1

Compared with trajectory retrieval, revised SAGA improves Look by 27.8 points, Pick2 by 23.5 points, and overall performance by 12.6 points. These results provide direct evidence of transfer to held-out task types under a non-saturated backbone. Revised Table 5 reports this controlled comparison and limits the transfer conclusion to the fixed Qwen2.5-7B setting.

W4, W5 & Q4: ALFWorld configurations and backbone matching

The 45.5 → 57.5 result in the submission referred to the Qwen2.5-7B trajectory-only control, but its description did not make the configuration sufficiently clear. The complete mapping is:

ActorConfigurationOverall
Qwen2.5-7BNo memory45.5
Qwen2.5-7BTrajectory-only57.5
Qwen2.5-7BSubmitted complete hierarchy59.4
Qwen2.5-7BRevised SAGA70.1
Qwen3.8-MaxNo memory96.3
Qwen3.8-MaxSAGA99.3

The “Base Model” row is Qwen2.5-7B-Instruct without memory. The revised manuscript distinguishes the 57.5% trajectory-only result, the 59.4% submitted complete-hierarchy result, and the 70.1% outcome-validated result under the same fixed actor. Tables 4 and 5 now identify the actor and configuration for every SAGA row.

The prior-work results in the submission came from the original papers and used different backbones or evaluation protocols. The revised manuscript separates these reported cross-paper results from matched evaluations. The controlled comparisons use the same actor, source experiences, split, interaction budget, context budget, and evaluation harness.

AuthorsOpenReview ↗

Response to Reviewer G7KD (Part III)

W6 & Q5: Same-backbone comparisons with experience-based methods

We thank the reviewer for identifying this missing comparison. We conducted matched evaluations by reimplementing each method according to its paper description, prompts, and algorithm. Within each benchmark, all methods use the same actor, source experiences, task split, interaction budget, context budget, and evaluation harness.

On ScienceWorld, all methods use Qwen3.8-Max:

MethodAvg. Score (mean ± SD)
No memory48.4 ± 0.5
Trajectory retrieval63.7 ± 0.6
ExpeL65.2 ± 0.9
Memento66.4 ± 1.0
ReasoningBank66.7 ± 0.8
CLIN66.9 ± 0.9
Submitted SAGA67.0 ± 0.7
Revised SAGA72.4 ± 1.1

Submitted SAGA performs comparably to the strongest matched baseline. Revised SAGA exceeds CLIN by 5.5 points (95% CI: [4.1, 6.9], p = 0.002).

On ALFWorld, all methods use Qwen2.5-7B as the actor and Qwen3.8-Max for memory or rule construction. Look and Pick2 remain excluded from experience collection:

MethodOverall success rate
No memory45.5
Trajectory-only57.5
AutoGuide61.9
AutoManual64.1
Revised SAGA70.1

Revised SAGA exceeds AutoGuide by 8.2 points and AutoManual by 6.0 points. Revised Tables 3 and 6 report these matched comparisons, and the Appendix specifies the backbone assignments, source experiences, task splits, budgets, and evaluation settings.

W7: Source of the SFT data and the roles of distillation and RL

We thank the reviewer for raising this attribution question. For ScienceWorld, the SFT corpus is reconstructed from Qwen3.8-Max trajectories collected during prompt-based evolution. Full-score trajectories contribute all stepwise action targets; zero-final-score trajectories contribute the prefix to the first scoring peak; and partial-score trajectories contribute the prefix ending at the highest attained score. For ALFWorld, the teacher corpus contains successful Qwen3.8-Max trajectories from the four seen task types. Qwen2.5-7B is trained on the selected teacher actions together with their hierarchical knowledge context. This stage contains a substantial teacher-action distillation component.

GRPO starts from the SFT checkpoint and optimizes Qwen2.5-7B through new on-policy rollouts over the training task pool. We added matched controls to separate teacher-action distillation, hierarchical knowledge conditioning, and builder strength:

Qwen2.5-7B configurationKnowledge builderSFTSFT + GRPO
Same teacher actions, no knowledge contextNone53.862.7
Same-model SAGA knowledgeQwen2.5-7B55.163.8
Full SAGA knowledgeQwen3.8-Max58.166.9

All variants use the same teacher trajectories and action targets. The GRPO variants additionally share the same task pool, reward, rollout budget, and optimization settings. Teacher-action distillation raises the base score from 9.6 to 53.8 after SFT. With teacher supervision fixed, full hierarchical knowledge conditioning adds 4.3 points after SFT and 4.2 points after matched GRPO. The same-model builder also provides positive gains of 1.3 and 1.1 points, respectively. These controls separate the large contribution of teacher-action distillation from the additional contribution of knowledge construction and utilization.

Revised Table 2 reports these matched controls. Section 4.1 and the Appendix now state the data provenance, trajectory-selection rules, builder/actor assignments, and the distinction between SFT teacher trajectories and GRPO on-policy rollouts. The 13.9 score in Table 1 is identified as a previously reported result, while 9.6 is the matched initialization measured with our training-based evaluation harness.

W8: Evaluation protocol and checkpoint selection

The 66.9% checkpoint was selected without using test performance. During GRPO, checkpoints were ranked only by their average reward on the training task pool. The checkpoint with the highest training reward was fixed before test evaluation and evaluated once on the held-out test set. No separate validation split was introduced.

CheckpointSelection ruleTest performance
Training-selected checkpointHighest average reward on the training task pool66.9%
Final checkpointLast GRPO checkpoint66.2%

Figure 4 and the accompanying analysis now use the label “Training-selected checkpoint,” state the selection criterion, and report the 66.2% final-checkpoint result. The revised manuscript also distinguishes the previously reported 13.9 result from the 9.6 matched initialization used in the training-based experiments.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of Review Feedback and Rebuttal Discussion

Dear Area Chair and Reviewers,

Thank you for the time and care you have given our submission. During the discussion, Reviewer G7KD raised their rating from 2 to 3 (Weak acceptance). Reviewer 8dRT noted that the additional analyses alleviated their concerns and suggested a further experiment in the main ScienceWorld setting. We appreciate that suggestion and summarize the result below, together with the main points from each review.

Reviewer G7KD. We appreciate the reviewer’s distinction between the results of submitted SAGA and the gains from outcome validation. The reviewer identified several issues as resolved, including checkpoint selection, predicate compilation, and the ceiling effect on ALFWorld. On ScienceWorld, submitted SAGA scored 67.0 versus 63.7 for trajectory retrieval, with a positive difference in all five paired runs. Its performance was comparable to our fixed-experience CLIN-style baseline at 66.9. Revised SAGA reached 72.4 using additional training rollouts for outcome validation. In our follow-up, we clarified how the CLIN baseline updated its memory, which budgets were shared, and how SAGA used rollout outcomes to identify harmful principles.

Reviewer 8dRT. The reviewer asked whether principle-level knowledge still helps when procedures receive the same outcome validation. We ran this comparison with the same frozen actor, five memories, validation variants, rollout budget, action regulation, and test instances. Validated L2 procedures scored 68.5 ± 1.0, while validated L3 principles scored 72.4 ± 1.1. The reviewer’s suggestion made it possible to compare the two forms of knowledge directly under the validation procedure used in the final system.

Reviewer Zxva. The reviewer highlighted the finding that inserting principles verbatim can hurt while adapting them to the current task helps. Their questions prompted us to examine how reliably this finding holds, how well the learned knowledge transfers, and how much the stronger knowledge builder contributes. With a frozen 7B actor, revised SAGA improved over trajectory retrieval on both held-out ALFWorld task types: from 50.0% to 77.8% on Look and from 47.1% to 70.6% on Pick2. With the teacher trajectories held fixed, the stronger builder accounted for 3.1 of the 4.2 points gained over the no-knowledge training control. As of this summary, our response is the latest update in that thread.

Together, the experiments clarify two aspects of learning from interaction. The original ablations showed that adapting a principle to the current task improved performance over inserting it verbatim on both ScienceWorld backbones. The follow-up comparison showed that, even with the same outcome-validation procedure, principles derived across procedures scored 3.9 points higher than the procedures themselves.

We thank the Area Chair and all three reviewers for their thoughtful feedback. Your questions helped us design more informative controls and explain more precisely where the gains come from.

Sincerely,
The Authors of Submission 814

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.