acceptodds
Under review as a conference paper at ICLR 2027

Beyond Positive Reinforcement: A Hierarchical Multi-Perspective Memory Framework for Multi-Agent Systems

Abstract

Recent advancements in Multi-Agent Systems (MAS) have witnessed significant progress in equipping agents with episodic memory, typically designed to store and retrieve successful collaborative trajectories. However, a critical design consideration remains insufficiently addressed in existing approaches: they predominantly rely on positive reinforcement, with insufficient utilization of the pedagogical value provided by failure experiences. Ignoring these error signals leads to a positive bias, making agents prone to local optima and less robust against cascading failures. To bridge this gap, we propose a Multi-perspective Memory (M-Memory) framework that draws inspiration from three empirically-grounded memory organization strategies. Concretely, M-Memory integrates both successful and failed memory records and structures them at two levels. At the high level, the framework synthesizes three types of insights: Shortcuts from successful tasks, Pitfalls from failures, and Cross-domain Consensus from diverse experiences. At the low level, it enriches key interaction steps with trajectories from both failures and varied successes to boost robustness and generalization. For a new task, M-Memory bidirectionally retrieves high-level insights and fine-grained trajectories to guide execution. By integrating new outcomes, this hierarchical feedback loop drives incremental MAS evolution. The superiority of our approach is demonstrated through extensive experiments across four benchmarks, two LLM architectures, and three prominent MAS.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Large Language Model Agents, Multi-Agent Systems, Hierarchical Memory, Learning from Failure

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_f9iYOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper mainly studies the problem of accumulating experience from failures into memory in multi-agent systems. In multi-agent systems (MAS), memory modules mainly store and retrieve successful experiences. Lessons from failures are not utilized, so the authors argue that agents develop a "positive bias", and get trapped in local optima.

The authors propose M-Memory, which divides memory into two levels. The high level distills three types of insights: shortcuts, traps, and cross-domain consensus. The low level condenses similar successful task trajectories into key steps, and uses a similar failed trajectory and a dissimilar successful trajectory as references. Retrieval uses cosine similarity to find similar tasks and MMR to find dissimilar tasks. Each agent receives memory tailored to its role.

They integrate this method into three MAS frameworks and use two LLMs, testing it on four benchmarks. Whether the memory is initially empty, or has accumulated experience over three rounds, this method outperforms all baseline methods, and its advantage becomes even greater after accumulation. Ablation experiments show that adding traps and consensus in sequence can progressively improve the results. They also report that the token cost of this method is lower than that of G-Memory.

The contribution presented in the paper is that it proposes a multi-perspective hierarchical memory framework for MAS, and experimentally demonstrates its advantages in performance and efficiency.

Critical strengths
  1. The framework proposed in the paper is plug and play. M-Memory does not require any changes to the structure of the MAS, and can be integrated into AutoGen, DyLAN, and MacNet, which respectively use centralized division of labor, debate-based, and decentralized structures.
  2. The method shows thoughtful consideration in its handling of failure signals. The authors analyze five possible causes of a failure in the paper: the insight itself was wrong, the agent did not follow it, environmental randomness, errors in multi-agent collaboration, and the insight was retrieved but never actually used. Only the first case should weaken the insight. So the authors choose not to reduce the support for an insight when a failure occurs, but instead route the failure information through several other proposed paths. This is a very practical credit assignment problem, and the authors provide clear reasoning and alternatives.
  3. Separately reporting the first-round results starting from empty memory is a good experimental setup. In the first round, each task can only use memories left by tasks completed before it, and cannot use its own trajectory. Under this setup, M-Memory outperforms the strongest baseline across all three frameworks and both backbones. This helps rule out the possibility of memorizing test questions, showing that the organization of memory itself has an effect.
Critical weaknesses
  1. I find the novelty to be somewhat limited, and the characterization of prior work is not entirely accurate. The concept of learning from failure has already been explored in works such as Reflexion, ExpeL, REMEMBERER/RLEM, and ReasoningBank. Furthermore, in the multi-agent domain, there is G-Memory. The paper’s claim that G-Memory relies too heavily on similar successful tasks is inaccurate; G-Memory’s query nodes record both successes and failures, generate insights from failed tasks, and utilize prompts that compare successful and unsuccessful trajectories. The overall framework also mirrors that of G-Memory—incorporating elements like a three-layer graph, hop-based expansion, trajectory summarization, LLM-based relevance scoring, bidirectional traversal, and role-based memory initialization. The actual innovations appear to be limited to the categorization of insights into three types, the use of MMR (Maximal Marginal Relevance) to retrieve dissimilar tasks, and the application of more conservative update rules alongside additional reference examples during trajectory summarization.
  2. The classification of insights into three types lacks a solid rationale and fails to account for dimensions specific to multi-agent systems. While the classification seems to be inspired by an analogy to neuroscience, essentially a 2x2 matrix based on similarity and success/failure (notably omitting "dissimilar-failure" tasks, without explanation), this core innovation is not tied to distinct underlying mechanisms. Moreover, there is no experimental validation for this specific taxonomy (while the paper includes incremental ablation studies, it lacks comparisons against alternative classification schemes or justifications for this particular approach). More importantly, although the paper emphasizes its multi-agent nature, the design does not effectively reflect this. The proposed method could just as easily be applied to a single-agent setting, the paper could place greater emphasis on the dynamics of collaboration and conflict among agents. As it stands, the approach treats collaborative failures as noise to be filtered out, rather than as valuable experiences to learn from.
  3. There is an overlap between the test set and the dataset used to construct the memory. The main experiments involve running three iterations over the same set of tasks, by the third iteration, the memory already contains trajectories and insights derived from those very tasks. This creates a risk of "memorizing" the test questions, leading to data leakage. Consequently, the authors' claims regarding generalization capability are not well-supported.
Questions for authors
  1. Is it possible to provide a check for whether any answers have leaked into the memory?
  2. Why is there no insight type for dissimilar failed tasks?
6 comments

W1. I find the novelty ... during trajectory summarization.

Thank the reviewer for the insightful comments. We do not claim that "learning from failure" is itself a novel contribution. ExpeL, RLEM, and Reflexion have already demonstrated the value of failure experiences, and in our experiments we therefore compare directly against Reflexion and RLEM as baselines (on AutoGen at Round-3, they score 57.92 and 53.20, respectively, while M-Memory scores 69.19). Our claim is not whether failure experiences are useful, but that existing methods still exhibit insufficient utilization of the pedagogical value provided by failure experiences: Failure experiences are used, but they are not treated as a category of retrieval objects that requires its own frame of reference and its own slots.

We frame this underutilization as an access problem rather than an inventory problem. Existing methods typically record memory categories—successful demonstrations, failure lessons, or transferable consensus—as metadata at write time, without making them first-class retrieval variables. All categories therefore compete in one candidate pool under a single relevance function for limited slots. When this function favors surface similarity, abstract or lexically dissimilar categories may rank systematically lower without being explicitly excluded. Without category-level outcome feedback, the retrieval policy struggles to detect or correct this bias. Failures thus remain in memory but out of reach, leaving the agent repeatedly exposed to similar experiences and a narrower range of perspectives.

M-Memory turns categories from textual attributes into retrieval control variables. Each category has its own candidate pool and ranking, so a failure lesson need not outrank a successful demonstration globally—only other lessons from similar failed tasks. We therefore argue that categorization must be built into the retrieval operator through four stages: candidate pool partitioning, slot allocation, within-category ranking, and credit assignment. Our framework support this view: incorporating categories into these stages enables successful, failed, and consensus experiences to be retrieved together for a single task, making multiple perspectives operational.

We will revise "G-Memory over-rely on similar successful tasks" to state that G-Memory anchors retrieval to a single relevance neighborhood and therefore makes insufficient use of multi-perspective experience. We will clarify in the contributions that M-Memory identifies and fixes a structural blind spot of hierarchical MAS memory, namely that evidence type appears only in the content of insights and never in the retrieval and credit operators.

W2#1 The classification of insights ...for this specific taxonomy (while the paper includes incremental ablation studies, it lacks comparisons against alternative classification schemes or justifications for this particular approach).

Q2.Why is there no insight type for dissimilar failed tasks?

We thank the reviewer for the comments. The rationale for our three-way classification of insights lies in "what information a memory can provide to the agent." Specifically, a successful memory tells the agent what to do in order to succeed, and a failed memory tells the agent what to do in order to avoid a particular failure mode. Accordingly, similar successes serve a reuse function, similar failures serve a warning function, and diverse successes can point to another success pattern that may be borrowed and can help the agent broaden its view. In contrast, diverse failures can only rule out one possible failure mode, and since the causes that can lead to failure are far more numerous than those that can lead to success, diverse failures are in most cases noise that provides no actionable information [1]. For this reason, we do not model the diverse-failure perspective of the agent.

To further verify the above view, we add an experiment in which diverse failure is injected as a new perspective, with the total number of insights held constant. The results on the PDDL dataset with Qwen2.5-14b are as follows.

Round 1 results

SettingMASPDDLPDDL (RE)
M-Memory with diverse failureAutoGen31.6751.00
DyLAN28.3354.81
MacNet26.6749.47
Full M-MemoryAutoGen36.6758.38
DyLAN31.6756.19
MacNet31.6753.45

Round 3 results

SettingMASPDDLPDDL (RE)
M-Memory with diverse failureAutoGen35.0056.09
DyLAN36.6760.92
MacNet28.3352.09
Full M-MemoryAutoGen43.3364.22
DyLAN41.6766.96
MacNet35.0055.43

The results show that after the diverse-failure perspective is injected, M-Memory shows degraded performance in both generalization and self-evolution capability, which is consistent with the view stated above. In addition, Appendix A.4.6 discusses the applicability boundary of consensus insights. A positive, concrete, and executable entry from a diverse sub-domain (tyreworld → blocksworld) still fails to take effect, and the cost is borne by prompt length and attention dilution. A negative entry from a diverse sub-domain would apply in even fewer scenarios and would additionally introduce misleading information.

[1] Gentner, D. (1983). Structure-mapping: A theoretical framework for analogy. Cognitive Science, 7(2), 155-170.

W2#2. More importantly, ... rather than as valuable experiences to learn from.

We thank the reviewer for this comment. We would like to clarify that, in our method, collaborative failures are not discarded as noise to be filtered out. Instead, they are explicitly exploited as learnable experience. First, in the memory writing stage of M-Memory, we designed a dedicated pathway. This pathway simultaneously retrieves similar failure trajectories and diverse success trajectories, and injects both into the prompt for key-step extraction, requiring the model to “refer to the failure trajectories and filter out erroneous actions such as Nothing happens”(Eq. 7). Failure trajectories therefore contribute directly to the formation of experience rather than being excluded from it. Second, Appendix A.3 provides a dedicated analysis of collaborative failures. We attribute them to three categories, namely role confusion, communication conflict, and collaborative deadlock, and we explain that M-Memory routes them through hyperedge binding, rule-level auditing, and counterexample trajectory extraction, so that different types of collaborative failures enter different memory structures.

At the same time, we acknowledge the limitation pointed out by the reviewer. The current failure routing mechanism does learn from collaborative failures, yet it does not learn who caused them. Our failure attribution remains at the task level and is not grounded in a specific agent or a specific interaction pattern, and collaborative failure is neither explicitly modeled nor analyzed in isolation. In the revised manuscript, we will discuss the collaboration dynamics and conflicts among agents more explicitly and will list the explicit modeling and separate analysis of collaborative failures as an important direction for future work.

Reviewer_btjiOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

This paper proposes M-Memory, a hierarchical multi-perspective memory framework for multi-agent systems (MAS). It organizes experience into three categories—Shortcuts, Pitfalls, and Cross-domain Consensus—and combines similarity-based retrieval, MMR-based diversity retrieval, and outcome-conditioned memory updates to improve multi-agent task execution. Experiments across multiple benchmarks, LLM backbones, and MAS frameworks show promising performance improvements.

Critical strengths
  1. The proposed framework provides a structured way to organize successful experiences, failure patterns, and cross-domain insights within a hierarchical memory architecture.
  2. The experiments cover multiple benchmarks and multi-agent frameworks.
Critical weaknesses
  1. Limited Novelty

Learning from successful and failed agent trajectories has been extensively studied. ExpeL (AAAI 2024) extracts insights from both successful and failed experiences, while ReasoningBank (ICLR 2026) develops reusable reasoning strategies through iterative retrieval and updates. Moreover, M-Memory's hierarchical graph structure and bidirectional retrieval closely resemble G-Memory [1]. The categorization into Shortcuts, Pitfalls, and Cross-domain Consensus, combined with MMR-based retrieval, appears to be an incremental combination of existing techniques rather than a substantial methodological contribution.

  1. Writing and Presentation Issues

The paper requires significant improvements in clarity and presentation. The methodology introduces multiple equations and retrieval mechanisms without adequately explaining their motivations or relationships. Several paragraphs are overly dense, and citation formatting is inconsistent. For example, "based on the Maximal Marginal Relevance (MMR) score Carbonell & Goldstein (1998)" incorrectly mixes narrative and parenthetical citation styles.

The experimental presentation also needs improvement. Tables 1 and 2 are overly dense, while Figure 3 contains overlapping text. Additionally, the paper should provide precise model identifiers (e.g., Qwen-2.5-14B and GPT-4o-mini) and corresponding references.

  1. Insufficient Evaluation of Generalization and Robustness

The evaluation covers only a limited set of backbone models, leaving the effectiveness on newer or stronger LLMs unclear. Experiments across models with different capability levels would better establish robustness. Furthermore, repeatedly evaluating the same tasks across three rounds may conflate genuine generalization with memorization or reuse of previous experiences. Evaluation on unseen tasks is needed to substantiate the claimed generalization and self-evolution capabilities.

[1] G-Memory: Tracing Hierarchical Memory for Multi-Agent Systems

Questions for authors

See weaknesses.

7 comments

W1.Limited Novelty Learning from successful ... substantial methodological contribution.

Thank the reviewer for raising this critical point. We do not claim that "learning from failure" is itself a novel contribution. ExpeL, RLEM, and Reflexion have already demonstrated the value of failure experiences, and in our experiments we therefore compare directly against Reflexion and RLEM as baselines (on AutoGen at Round-3, they score 57.92 and 53.20, respectively, while M-Memory scores 69.19). Our claim is not whether failure experiences are useful, but that existing methods still exhibit insufficient utilization of the pedagogical value provided by failure experiences: Failure experiences are used, but they are not treated as a category of retrieval objects that requires its own frame of reference and its own slots.

We frame this underutilization as an access problem rather than an inventory problem. Existing methods typically record memory categories—successful demonstrations, failure lessons, or transferable consensus—as metadata at write time, without making them first-class retrieval variables. All categories therefore compete in one candidate pool under a single relevance function for limited slots. When this function favors surface similarity, abstract or lexically dissimilar categories may rank systematically lower without being explicitly excluded. Without category-level outcome feedback, the retrieval policy struggles to detect or correct this bias. Failures thus remain in memory but out of reach, leaving the agent repeatedly exposed to similar experiences and a narrower range of perspectives.

M-Memory turns categories from textual attributes into retrieval control variables. Each category has its own candidate pool and ranking, so a failure lesson need not outrank a successful demonstration globally—only other lessons from similar failed tasks. We therefore argue that categorization must be built into the retrieval operator through four stages: candidate pool partitioning, slot allocation, within-category ranking, and credit assignment. Our framework support this view: incorporating categories into these stages enables successful, failed, and consensus experiences to be retrieved together for a single task, making multiple perspectives operational.

We will revise "G-Memory over-rely on similar successful tasks" to state that G-Memory anchors retrieval to a single relevance neighborhood and therefore makes insufficient use of multi-perspective experience. We will clarify in the contributions that M-Memory identifies and fixes a structural blind spot of hierarchical MAS memory, namely that evidence type appears only in the content of insights and never in the retrieval and credit operators.

W2.Writing and Presentation Issues The paper requires ... and corresponding references.

We thank the reviewer for the detailed comments on the writing and the typesetting. Regarding the motivation for the formulas and the retrieval mechanisms, and their interrelationships, we will begin Section 3 with an overview paragraph that describes the complete retrieval pipeline and the role that each mechanism plays in it. Each key formula will be preceded by a brief motivation statement that identifies the problem it addresses. In particular, diverse candidate generation ensures that the subsequent perspectives remain sufficiently broad (Eq. 5), memory retrieval must be able to explicitly locate the memories associated with different perspectives (Eq. 6), and memory updates must proceed independently for each category (Eqs. 16 and 17).

In the revised manuscript, we will correct the improper use of citation formats.

We will remove the "Venue" column from Tables 1 and 2 and report all numerical values to one decimal place.

For Figure 3, we will reposition the text within the figure to prevent overlap.

We will also label Qwen-2.5-14B and GPT-4o-mini explicitly and add the corresponding citations.

W3#1. The evaluation covers only a limited set of backbone models, leaving the effectiveness on newer or stronger LLMs unclear. Experiments across models with different capability levels would better establish robustness.

Thank the reviewer for the insightful suggestion. Following the setup of G-Memory, we use Qwen2.5-14B and GPT-4o-mini as backbone models. To evaluate performance on more recent models, we add Qwen3-14B as an additional backbone model. Due to time constraints, we have not yet completed all experiments, so we can currently report results only on the PDDL and ALFWorld datasets. These results show that Qwen3-14B achieves better overall performance on the AutoGen framework than on the DyLan framework, and that M-Memory maintains its performance advantage on Qwen3-14B. We will provide the remaining results in the subsequent revision.

Round 1 results with Qwen3-14B.

MASMethodPDDLPDDL (RE)ALFWorld
AutoGenNo-memory20.0035.8179.10
G-Memory33.3354.9485.82
M-Memory41.6760.1987.31
DyLANNo-memory3.3310.2276.87
G-Memory18.3334.9383.58
M-Memory20.0040.1087.31

Round 3 results with Qwen3-14B.

MASMethodPDDLPDDL (RE)ALFWorld
AutoGenNo-memory20.0034.6080.60
G-Memory38.3355.9188.06
M-Memory46.6763.5591.04
DyLANNo-memory5.0013.0076.12
G-Memory25.0041.8185.82
M-Memory33.3349.9289.55
Reviewer_GS54OpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

M-Memory organizes successful and failed multi-agent experiences into task records, interaction traces, and reusable insights. It retrieves similar successes, similar failures, and diverse successes to guide new tasks, then updates memory after execution. Experiments cover four benchmarks, three multi-agent frameworks, and two language models. The paper reports both repeated-task learning and a separate held-out evaluation.

Critical strengths
  1. The ablations support learning from failures and diverse successes together. For AutoGen, Table 3 reports average scores of 59.38 with Shortcuts only, 64.45 after adding Pitfalls, and 69.19 for the full system; the other frameworks show the same ordering (page 8). Expanding G-Memory's context budget does not close the gap, suggesting that prompt length alone does not explain the improvement.

  2. The held-out results extend the evidence beyond repeating the same tasks. Table 8 reports higher scores than G-Memory in 14 of 15 comparisons, with one tie (page 18). The provided prompts and discussion of irrelevant retrieved advice also help readers understand how the method works and where it can fail.

Critical weaknesses
  1. The gains do not yet establish cross-domain transfer. Retrieving semantically diverse tasks does not ensure that they come from different domains. The positive cases do not identify where the Consensus insights originated, and Table 3 changes both insight retrieval and trajectory processing together. The improvement could therefore come from additional useful same-domain advice or better summarization. Identifying the source tasks and separating these effects would make the explanation more convincing (pages 8 and 20–21).

  2. The held-out protocol needs more detail. The 60/40 split is specified, but it is unclear whether test outcomes enter memory, whether test tasks are repeated, or how results vary across splits. Reflexion also uses a different retry schedule that is not clearly mapped to the table results. Three independent runs and the reported tests provide useful evidence, but a nonsignificant difference should not be described as equivalence (pages 14 and 16–18).

Questions for authors
  1. Is memory frozen during Table 8's held-out evaluation, and are test tasks evaluated only once? Can you report results across independent splits and explain how Reflexion's retry schedule and feedback budget match the comparisons?

  2. Which source domains produced the Consensus insights in the positive cases? Can you separate the benefit of genuinely cross-domain examples from diverse same-domain examples, and the benefit of insights from trajectory summarization?

5 comments

W1. The gains do not yet ... more convincing (pages 8 and 20–21).

Q2. Which source domains produced ... trajectory summarization?

Thank the reviewer for the insightful comments. The term "domain" in our method indeed refers to distribution shift at the level of task instances or semantic clusters, rather than to sub-domain labels of datasets. Consequently, the support set of a Consensus insight is very likely to consist merely of multiple task instances from the same domain that have different initial states or goals. We will therefore narrow the claim of cross-domain diversity to cross-task diversity.

For the two positive cases provided in Appendix A.4, the target task of Case 4 is to judge whether "Tool has won three Oscars." is correct, and the Consensus insight cited by Case 4 is summarized from the task "Tim Roth is an English actor." The target task of Case 5 is "put some saltshaker on cabinet.", and the Consensus insight cited by Case 5 is summarized from the task "put a saltshaker in drawer." Since the FEVER dataset contains only judgment tasks, strictly speaking, both Consensus insights and the target tasks belong to the same sub-domain.

To separate the benefit of genuinely cross-domain examples from that of diverse same-domain examples, we analyzed the ALFWorld experiments on AutoGen. Of the 122 successful experiments in total, only 4 actually cited insights generated from cross-sub-domain tasks.

To separate the benefit of insights from that of trajectory summarization, we added the following ablation experiment, namely M-Memory without trajectory summarization(Qwen2.5-14B). The experimental results show that trajectory summarization prompts have a strong positive effect on the agent's task execution. We will include these experimental results in the revised manuscript.

M-Memory w/o trajectory summarization

MASALFWorldPDDLFEVERHotpotQA
AutoGen75.3745.28/20.0073.3360.33
Dylan74.6349.56/28.3369.3354.67
MacNet67.1647.64/25.0067.6753.33

Complete M-Memory

MASALFWorldPDDLFEVERHotpotQA
AutoGen91.0443.33/64.2278.6768.67
DyLAN87.3141.67/66.9675.6759.67
MacNet84.3335.00/55.4373.6763.00

W2.#1 The held-out protocol ...across splits.

Q1.Is memory frozen ... match the comparisons?

We thank the reviewer for the suggestion. In the held-out evaluation, the training set contains 60% of the tasks in the entire dataset, and the test set contains 40% of the tasks in the entire dataset. Memory is not frozen during training or testing. The agent runs continuously on the training set three times in order to construct a mature memory architecture, and is then evaluated once on the test set. At test time, the agent's memory is still summarized and adjusted according to the test task. We randomly shuffled the tasks in the training and test sets and conducted a new 60/40 held-out evaluation on this basis. The table below shows that different data splits affect the experimental results to some extent. We believe that the main reason lies in the relatively small size of the datasets, which makes the evaluation metrics sensitive to the specific composition of the training and test sets, so that a change in the split introduces a certain amount of variation in the results. Nevertheless, the overall trend of the results remains consistent across splits, and the conclusions in the main text therefore do not depend on any particular split. We will include the results of the new held-out evaluation in the Appendix:

Held-out results across different splits (M-Memory):

MASDatasetsplits1splits2
AutoGenALFWorld0.8710.852
FEVER0.7330.720
HotpotQA0.4330.475
PDDL0.3610.350
PDDL (RE)0.5810.572
DyLANALFWorld0.8520.849
FEVER0.7000.710
HotpotQA0.3500.333
PDDL0.2920.317
PDDL (RE)0.6190.578
MacNetALFWorld0.8520.871
FEVER0.7100.690
HotpotQA0.4170.430
PDDL0.3330.292
PDDL (RE)0.6310.656

W2.#2 Reflexion also uses a different retry schedule that is not clearly mapped to the table results.

We thank the reviewer for pointing out this ambiguity. The retry schedule that executes the same task three times consecutively is the schedule adopted in the original Reflexion paper. In fact, we examined two retry schedules for Reflexion, and in the main text we reported only the one that performed better. Below we describe the definitions and results of the two schedules, and then explain how we will present them in the revised manuscript.

Schedule 1: a cross-task schedule fully aligned with the baselines In this setting, all methods execute tasks sequentially under the same protocol, and the current task may draw on the reflections summarized from preceding tasks, so the reflections of Reflexion can be reused across tasks. Tables R1 and R2 report the third-round results under this setting.

Table R1. Third-round results of Reflexion under Schedule 1 (Qwen2.5-14B)

MASALFWorldPDDLPDDL (RE)FEVERHotpotQAAvg
AutoGen85.82±1.2920.00±2.8944.58±2.4765.00±1.0056.00±1.7354.28
DyLAN64.18±1.2925.00±2.8946.18±2.4863.33±1.5354.33±1.5350.60
MacNet58.21±1.2918.33±2.8945.00±2.5767.33±1.1554.00±1.0048.57

Table R2. Third-round results of Reflexion under Schedule 1 (GPT-4o-mini)

MASALFWorldPDDLPDDL (RE)FEVERHotpotQAAvg
AutoGen83.33±0.862.78±2.5521.58±1.8461.33±1.1555.33±1.5344.87
DyLAN60.45±1.496.67±1.679.18±1.2865.33±1.5359.67±2.0840.26
MacNet67.16±1.293.33±2.8917.30±2.0567.33±1.5353.67±1.1541.76

The results show that when Reflexion follows the same schedule as the other baselines, most of its third-round results are worse than its first-round results. We attribute this to the fact that the reflections of Reflexion are tightly bound to the specific task that produced them, so they often do not apply when transferred across tasks and may even introduce misleading experience. Appendix A.1.2 documents the round-by-round results that support this interpretation. In other words, this schedule does not provide Reflexion with the reusable feedback that its mechanism assumes. Instead, it systematically weakens the method.

Schedule 2: a schedule that executes the same task three times consecutively Motivated by this observation, we instead place Reflexion on the same task for three consecutive runs, so that its reflections always act on the task that produced them. Specifically, we no longer build a global memory for Reflexion. We call it three times consecutively on each task, and the task-specific reflections persist only across these three calls. For the reflections of Reflexion itself, we inject them in the form of insights and cap their number at five, as we do for the other baselines, so as to keep the budget as comparable as possible across methods.

Table R3. Third-round results of Reflexion under Schedule 2 (Qwen2.5-14B)

MASALFWorldPDDLPDDL (RE)FEVERHotpotQAAvg
AutoGen85.82±1.2933.33±2.8949.14±2.4765.33±1.0056.00±1.7357.92
DyLAN85.07±1.4931.67±2.8949.41±2.4863.33±1.5349.33±1.5355.76
MacNet67.91±1.2925.00±2.8949.19±2.5767.33±1.1554.00±1.0052.69

Table R4. Third-round results of Reflexion under Schedule 2 (GPT-4o-mini)

MASALFWorldPDDLPDDL (RE)FEVERHotpotQAAvg
AutoGen83.33±0.8610.00±3.3323.99±1.9362.67±2.0854.33±1.5346.86
DyLAN83.33±0.8611.11±0.9619.15±1.1764.67±1.5342.67±1.5344.19
MacNet67.16±1.499.44±2.5532.59±1.8164.00±1.7353.00±1.0045.24

To eliminate the ambiguity entirely, we will report the results of Reflexion under both schedules in the revised manuscript and will state explicitly in the table captions which schedule each dataset corresponds to. The main table will retain the better of the two sets of results, and the full results of the other set will be given in the appendix.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.