acceptodds
Under review as a conference paper at ICLR 2027

Attention-Derived Memory-Use Traces for Agent Memory Refinement

Abstract

Self-evolving agent memory is usually updated from task outcomes, execution trajectories, and post-hoc textual reflections. These signals can guide memory writing, but they provide limited evidence about how retrieved memory was actually used in the decisions made during execution. Without such read-side evidence, it is difficult to distinguish among different causes of memory-related errors. We introduce execution-time memory utilization as a complementary signal for persistent memory refinement. To estimate this utilization, we turn to attention as an indicator of memory access. Because raw attention is distributed across layers and heads, remains at the token level, and is not aligned with semantic memory segments or agent decisions, we identify model-specific retrieval heads and aggregate their attention into a segment-by-decision memory-use trace. Intervention experiments provide temporal and segment-level evidence that this trace reflects memory utilization. We then apply this signal in Attention-Guided Memory Refinement (AGMR), a simple refinement procedure that compares the utilization expected from reflection with the utilization observed in the trace to propose targeted edits. Across three interactive benchmarks and two models, AGMR improves task performance while generally reducing memory length.

Primary area:
foundation or frontier models, including LLMs
Keywords:
LLM agents, agent memory, self-evolving agents, memory refinement, memory utilization, mechanistic interpretability

Reviews

Among:
Shape:
1234this paper, Top 17%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.75 over 4 reviews, above 83% of the others.
Reviewer_2VfrOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies memory refinement for LLM agents from a read-side perspective. The key idea is to use execution-time attention patterns to estimate how retrieved memory segments are actually utilized during decision making. The authors first identify a small set of retrieval-sensitive attention heads from transition cases where adding trajectory memory changes task outcomes from failure to success. Their attention is then aggregated into a segment-by-decision memory-use trace. Based on the discrepancy between semantically expected memory use and observed attention-derived utilization, the method categorizes failures into accessed-but-ineffective, missed-memory, and distraction cases, and applies different textual memory-editing strategies. The resulting AGMR framework is evaluated on ALFWorld, ScienceWorld, and WebShop.

Critical strengths
  1. The paper introduces a useful read-side perspective on agent memory refinement. Existing memory refinement methods mainly rely on task outcomes, trajectories, and post-hoc semantic reflection, while this work explicitly incorporates information about how memory is accessed during execution. The idea of connecting memory reading and persistent memory writing is conceptually interesting.

  2. The paper makes a serious effort to validate the attention-derived signal. The intervention experiments provide evidences that the selected heads are functionally related to memory-assisted decision making rather than being arbitrary correlates.

  3. The empirical evaluation is reasonably comprehensive within the studied setting. AGMR consistently improves over trajectory-memory and text-only refinement baselines across multiple agent environments and two open-weight backbones, while often reducing memory length. The factor-matched text-only and random-head controls are particularly useful for isolating the contribution of the utilization trace.

Critical weaknesses
  1. The necessity of the attention-based failure classification is not sufficiently established. The main novelty of the paper lies in identifying retrieval-sensitive heads and using their utilization trace to distinguish among several failure modes. However, after this diagnosis, all branches ultimately reduce to textual memory rewriting: correcting misleading content, increasing the salience of relevant content, or clarifying competing memories. The current experiments show that adding the utilization trace improves over the authors' text-only control, but they do not establish that explicitly classifying a failure into these cases is necessary. The current results demonstrate that attention-conditioned refinement is useful under the proposed updater, but do not yet show that the proposed diagnosis-and-routing mechanism is itself necessary or offers a fundamental advantage over a stronger diagnosis-free rewriter.

  2. The reliability and generality of the identified retrieval heads remain unclear. The heads are selected from relatively small LLM-annotated supervision sets (47 cases for Qwen and 34 for Llama). Although the paper provides bootstrap stability and cross-benchmark transfer evidence, it is still unclear how robust the selected heads are to different supervision data, annotation procedures, memory structures, or model families.

Questions for authors
  1. Could the authors comment on how AGMR would compare with a more flexible unified memory rewriter that can jointly revise relevant and potentially conflicting memories without explicitly assigning one of the three failure types?

  2. It would also be useful to better understand the robustness of retrieval-head selection. Have the authors observed similar head sets or downstream performance when changing the supervision subset or the analyzer used to identify supporting memory segments?

2 comments

Response to Reviewer 2Vfr

We sincerely thank the reviewer for recognizing the read-side perspective and the factor-matched controls, and for the constructive questions about explicit diagnosis and retrieval-head robustness.

W1 and Q1: Necessity of Three Explicit Failure Categories

Our existing text-only control already removes both the memory utilization trace and the explicit three-category diagnosis. Its comparison with AGMR, together with the intervention results in Figure 5 and the case studies in Appendix F (Figures 12 and 13), demonstrates the effectiveness of attention-guided, diagnosis-based memory refinement.

So we understand the reviewer's question as whether explicitly classifying failures is necessary when the memory utilization trace is already available. In other words, could a unified rewriter directly use the attention-derived utilization trace to jointly revise relevant and potentially conflicting memory segments, without explicitly assigning one of the three failure categories?

To investigate this, we introduce an Attention-conditioned Unified Rewriter and compare it with our existing Text-only control and AGMR:

MethodMemory Utilization Trace UUExplicit Three-category Diagnosis
Text-only✗✗
Attention-conditioned Unified Rewriter✓✗
AGMR (Ours)✓✓

The Attention-conditioned Unified Rewriter retains the same two-stage pipeline and Semantic Reflector as AGMR. The only modification is in the Refiner: instead of explicitly classifying the failure and applying category-specific editing strategies, we directly provide the full memory utilization matrix UU to the LLM and allow it to freely determine how to jointly revise the memory.

The following comparison uses Qwen2.5-7B-Instruct on ALFWorld and ScienceWorld.

MethodALFWorld Success (%) ↑ScienceWorld Avg. Reward (×100) ↑
Text-only76.1372.85
Attention-conditioned Unified Rewriter75.3770.93
AGMR (Ours)81.3477.20

As shown above, the Attention-conditioned Unified Rewriter performs substantially worse than AGMR and even falls below the text-only control. This is perhaps unsurprising: providing an LLM with a raw utilization matrix alone does not necessarily tell it how to interpret the attention patterns or translate them into effective memory edits. Without an explicit diagnostic framework and corresponding editing strategies, the utilization trace may be underutilized or even introduce distracting signals.

In contrast, AGMR transforms the use of memory utilization traces from generation to discrimination. By explicitly classifying the observed attention patterns and mapping each category to a corresponding editing strategy, AGMR substantially reduces the burden on the LLM to interpret raw read-side evidence and independently derive appropriate editing actions. This structured diagnosis mechanism makes the utilization trace more actionable, leading to more effective and efficient memory refinement than unrestricted, attention-conditioned rewriting.

W2 and Q2: Robustness of Retrieval-Head Selection

We thank the reviewer for asking about the robustness of retrieval-head selection to the supervision data and analyzer.

Human verification. The original supervision sets, containing 47 Qwen cases and 34 Llama cases, were manually checked by the authors and co-authors. Ambiguous or incorrect annotations were excluded or corrected before head selection.

Stability experiments already included in the paper. The original manuscript evaluates retrieval-head selection with 10,000 bootstrap resamples and 1,000 random disjoint half-splits. Bootstrap Jaccard measures overlap with the full-data Top-5 set, while half-split Jaccard and overlap counts compare the Top-5 sets selected independently from two non-overlapping halves of the supervision data.

Stability metricQwenLlama
10,000 bootstrap resamples: mean Jaccard0.8670.811
1,000 disjoint half-splits: mean Jaccard0.7390.662
Disjoint half-splits: mean number of shared heads4.22/54.01/5

To further address the reviewer's questions, we additionally tested a different analyzer and three new supervision sets, as detailed below.

Changing the analyzer. We replaced the supporting-segment analyzer with GPT-5.4, keeping the original cases fixed. GPT-5.4 re-annotated the supporting memory segments without seeing the original labels. For both Qwen2.5-7B and Llama-3.1-8B, the selected Top-5 heads and their rankings remained unchanged.

Changing the supervision set. We additionally randomly sampled three mutually disjoint sets of 47 ALFWorld training cases, all without overlap with the original Qwen supervision set. Reselecting Qwen's retrieval heads from each of the three sets recovered exactly the same Top-5 head set as the original selection, with only their relative rankings changing.

BackboneSupervision dataAnalyzerTop-5 overlap with original selectionHead ranking
Qwen2.5-7B-InstructOriginal 47 casesGPT-5.45/5Unchanged
Llama-3.1-8B-InstructOriginal 34 casesGPT-5.45/5Unchanged
Qwen2.5-7B-InstructNew set 1: 47 casesGPT-5.45/5Changed
Qwen2.5-7B-InstructNew set 2: 47 casesGPT-5.45/5Changed
Qwen2.5-7B-InstructNew set 3: 47 casesGPT-5.45/5Changed
Reviewer_fMC9OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

this paper focuses on that self-evolving agent memory is updated from outcomes, trajectories, and post-hoc reflection, but such write-side evidence cannot tell how retrieved memory was actually read during execution. then the proposal selects retrieval-sensitive attention heads, aggregates their attention into a segment-by-decision memory-use trace, and uses it with a reflector to localize and a refiner compares expected with observed utilization. ‌incremental novelty seems to use attention as persistent read-side evidence for memory writing, i.e. bringing retrieval-head research (Wu et al., ICLR 2025) into self-evolving agent memory.

Critical strengths

the read-side angle is, imo, the strongest part. Prior memory writers (e.g. Reflexion, Shinn et al., NeurIPS 2023) work from text evidence only, so a mechanistic reading signal is a sensible missing piece.

presentation. the controls in Table 3 isolate the attention signal from pipeline effects, and Figure 9 shows where the gains actually come from.

Critical weaknesses

no uncertainty/statistic metrics on the main results. Tables 2 and 3 report three-run means only, and key margins are moderate, e.g. 81.34 vs 76.13 over the text-only control on Qwen-ALFWorld. i've also checked the appendix, where the reflector and refiner run at temperature 1.0 in your appendix, so construction noise is a real concern. Please add std plus paired tests against the Table 3 controls.

no memory baselines? The comparison has ReAct (Yao et al., ICLR 2023), ICL, AWM, and DC, while dedicated memory systems like MemoryBank (Zhong et al., AAAI 2024), Mem0 (Chhikara et al., 2025), and ReasoningBank (Ouyang et al., 2025) are absent, and they are already cited in Appendix A. Learned read-side critique, e.g. Self-RAG (Asai et al., ICLR 2024), is not positioned either. Add at least two memory systems on the same benchmarks or explain the choice. the recommendations may not be that appropriate, so the authors needs to double-check and add some comparisons.

the current reproducibility depends on a closed model. The reflector and refiner are gemini-3.1, and head supervision is 47 qwen plus 34 llama-labeled cases. release the labels, spot-check a subsample, and rerun with an open-weight refiner, may help.

causal audit covers one setting? all decision-level evidence is qwen alfworld, and your appendix e7 or e6 admits the pairs are not replicated. Llama half-split overlap is 4.01 of 5 heads in your appendix, e.g. shakier head stability. so it's required to repeat the temporal intervention on llama and one more environment.

Questions for authors

see above weakness

3 comments

Response to Reviewer fMC9

We sincerely thank the reviewer for recognizing the read-side contribution and the factor-matched controls. We appreciate the request to quantify memory-construction variability.

W1: Construction Variability

Standard deviations. We have added across-construction standard deviations to Tables 2 and 3; the complete updated tables are provided below.

Temperature setting. We use temperature 1.0 for the Semantic Reflector and Refiner following Google's official Gemini 3 guidance. Google recommends retaining the default temperature of 1.0 because Gemini 3's reasoning capabilities are optimized for this setting; reducing it can lead to degraded reasoning performance. Regarding the paired test. In the available single-run Qwen–ALFWorld comparison against the 75.37% control, AGMR achieves a net gain of 8/134 tasks (15 wins vs. 7 losses; +5.97 percentage points; two-sided exact McNemar p = 0.134).

Updated Table 2. Performance comparison across environments and backbone models. For clarity, we report the results for Qwen2.5-7B-Instruct and Llama-3.1-8B-Instruct separately

Qwen2.5-7B-Instruct

EnvironmentMetricReActICLAWMDCTrajACEReasoningBankAGMR
ALFWorldPerformance ↑41.79 ± 045.52 ± 064.18 ± 0.4352.24 ± 0.7570.90 ± 059.70 ± 0.8652.24 ± 0.4381.34 ± 0.75
ALFWorldAvg. steps ↓28.40 ± 026.71 ± 022.22 ± 0.3525.09 ± 0.4619.36 ± 022.39 ± 0.3725.45 ± 0.0416.08 ± 0.15
ALFWorldMemory tokens ↓——312.01 ± 4.281013.73 ± 8.12852.36 ± 015929.00 ± 48.80145.97 ± 5.64752.53 ± 7.22
ScienceWorldPerformance ↑29.88 ± 029.38 ± 050.39 ± 1.3363.73 ± 0.4167.69 ± 044.27 ± 0.8030.80 ± 0.1877.20 ± 0.71
ScienceWorldAvg. steps ↓19.56 ± 018.35 ± 016.92 ± 0.1515.52 ± 0.0614.62 ± 018.34 ± 0.2719.14 ± 0.4313.78 ± 0.15
ScienceWorldMemory tokens ↓——435.32 ± 8.661478.85 ± 39.511040.62 ± 015974.64 ± 62.14158.03 ± 4.37990.89 ± 7.09
WebShopPerformance ↑56.66 ± 060.12 ± 056.11 ± 0.4454.04 ± 0.5951.04 ± 057.70 ± 1.0459.36 ± 0.8963.86 ± 0.57
WebShopMemory tokens ↓——185.65 ± 9.25618.14 ± 12.451372.22 ± 014042.51 ± 128.16180.75 ± 6.92503.77 ± 8.80

Llama-3.1-8B-Instruct

EnvironmentMetricReActICLAWMDCTrajACEReasoningBankAGMR
ALFWorldPerformance ↑0.00 ± 01.49 ± 07.46 ± 0.4311.19 ± 0.4326.12 ± 020.15 ± 1.7218.66 ± 0.8647.76 ± 0.86
ALFWorldAvg. steps ↓40.00 ± 039.74 ± 038.30 ± 0.2436.67 ± 0.0332.54 ± 034.66 ± 0.3735.23 ± 0.4925.52 ± 0.39
ALFWorldMemory tokens ↓——312.01 ± 4.281133.51 ± 17.07852.36 ± 015984.78 ± 213.29140.13 ± 6.18845.91 ± 5.82
ScienceWorldPerformance ↑26.50 ± 034.97 ± 052.14 ± 0.7672.28 ± 0.8375.63 ± 056.88 ± 0.2450.21 ± 0.9983.74 ± 0.45
ScienceWorldAvg. steps ↓19.59 ± 018.37 ± 016.23 ± 0.1213.99 ± 0.2513.22 ± 015.51 ± 0.3316.84 ± 0.2512.04 ± 0.11
ScienceWorldMemory tokens ↓——435.32 ± 9.961168.99 ± 22.591040.62 ± 014825.59 ± 83.79164.93 ± 8.58918.53 ± 19.90
WebShopPerformance ↑33.06 ± 052.10 ± 041.56 ± 1.5029.68 ± 1.6241.45 ± 049.29 ± 0.4357.86 ± 0.8363.26 ± 0.40
WebShopMemory tokens ↓——185.65 ± 7.82658.96 ± 25.111372.22 ± 014293.16 ± 58.72163.32 ± 7.90567.57 ± 34.99

Updated Table 3. Factor-matched attention controls.

BackboneRefinement settingAttention inputALFWorld: Performance / StepsScienceWorld: Performance / Steps
Qwen2.5-7B-InstructTwo-call text-onlyNone76.13 ± 0.43 / 17.36 ± 0.0472.85 ± 0.70 / 14.15 ± 0.17
Qwen2.5-7B-InstructRandom-5-headRandom heads75.37 ± 1.29 / 18.28 ± 0.1370.91 ± 1.28 / 14.38 ± 0.23
Qwen2.5-7B-InstructAGMRRetrieval heads81.34 ± 0.75 / 16.08 ± 0.1577.20 ± 0.71 / 13.78 ± 0.15
Llama-3.1-8B-InstructTwo-call text-onlyNone41.04 ± 0.75 / 28.06 ± 0.4779.72 ± 0.73 / 12.53 ± 0.13
Llama-3.1-8B-InstructRandom-5-headRandom heads38.81 ± 1.72 / 29.81 ± 0.2677.68 ± 0.21 / 12.79 ± 0.09
Llama-3.1-8B-InstructAGMRRetrieval heads47.76 ± 0.86 / 25.52 ± 0.3983.74 ± 0.45 / 12.04 ± 0.11

W2: Additional Memory Baselines and Related Work

We appreciate the reviewer's suggestions for broader baseline coverage. Agent Workflow Memory (AWM) and Dynamic Cheatsheet (DC) are both strong and influential baselines in experiential agent memory, with 849 and 245 citations to date, respectively. They were selected because they directly study the reuse and evolution of task-solving experience.

To address the reviewer's concern, we have added two additional recent memory baselines: ReasoningBank (364 citations) and ACE (569 citations). Their results have been incorporated into the updated Table 2 provided above, which includes the complete comparison with both newly added baselines. AGMR still achieves the highest task performance across all six backbone–environment configurations in the updated Table 2.

Regarding the other suggested systems, MemoryBank emphasizes long-term personalized interaction, and Mem0 focuses on extracting, consolidating, and retrieving information across extended conversations. Our benchmarks instead evaluate the refinement and reuse of task-solving experience for interactive actions. We therefore prioritized AWM, Dynamic Cheatsheet, ReasoningBank, and ACE as more directly aligned comparisons for this setting. We have also added Self-RAG (Asai et al., 2024) to Appendix A's Related Work and clarified its relationship to AGMR.

W3: Reproducibility, Label Quality, and an Open-Weight Refiner

We appreciate the reviewer's suggestions for improving reproducibility.

Data release. We have released all retrieval-head supervision data in our anonymous repository (https://anonymous.4open.science/r/AGMR_code-3262/): data/retrieval_dataset_qwen.json and data/retrieval_dataset_llama.json.

Spot-check. All candidate supervision cases were carefully reviewed by the authors and co-authors. Over 90% of the candidate annotations were consistent with our expert judgments; ambiguous or incorrect annotations were excluded or corrected. The released files contain the final screened cases used for retrieval-head selection.

To further assess the reliability of our retrieval-head supervision data, we tested whether the selected heads remain stable when changing the labeling model and resampling the training cases used to construct the supervision set.

Changing the analyzer. We replaced the labeling model of head supervision set with GPT-5.4, keeping the original cases fixed. GPT-5.4 re-annotated the 47 qwen plus 34 llama cases without seeing the original labels. For both Qwen2.5-7B and Llama-3.1-8B, the selected Top-5 heads and their rankings remained unchanged.

Changing the supervision set. We also randomly sampled three mutually disjoint sets of 47 ALFWorld training cases, all without overlap with the original Qwen supervision set. Reselecting Qwen's retrieval heads from each of the three sets recovered exactly the same Top-5 head set as the original selection, with only their relative rankings changing.

The results are summarized below.

BackboneSupervision dataAnalyzerTop-5 overlap with original selectionHead ranking
Qwen2.5-7B-InstructOriginal 47 casesGPT-5.45/5Unchanged
Llama-3.1-8B-InstructOriginal 34 casesGPT-5.45/5Unchanged
Qwen2.5-7B-InstructNew set 1: 47 casesGPT-5.45/5Changed
Qwen2.5-7B-InstructNew set 2: 47 casesGPT-5.45/5Changed
Qwen2.5-7B-InstructNew set 3: 47 casesGPT-5.45/5Changed

These results support the reliability of our supervision procedure and the robustness of head selection to both the labeling model and the sampled training cases. They complement the 10,000 bootstrap resamples and 1,000 random disjoint half-splits already included in the original manuscript, which also show stable head selection under resampling.

Open-weight refiner. Gemini-3.1 is still available through Google's official API. To further assess dependence on the refinement model, we additionally evaluate AGMR with the open-weight DeepSeek-V4-Pro as both the Semantic Reflector and Refiner, keeping the execution backbones, initial memories, retrieval heads, and refinement/verification protocol unchanged. The results are added below.

Execution backboneReflector and RefinerALFWorld Success (%) ↑ScienceWorld Avg. Reward (×100) ↑
Qwen2.5-7B-InstructTraj (no refinement)70.9067.69
Qwen2.5-7B-InstructGemini-3.1-Pro81.3477.20
Qwen2.5-7B-InstructDeepSeek-V4-Pro77.6175.89

Although DeepSeek-V4-Pro performs below Gemini-3.1-Pro, it improves over the unrefined Traj baseline by 6.71 percentage points on ALFWorld and 8.20 points in ScienceWorld average reward (×100). These results demonstrate that AGMR also provides gains with an open-weight Reflector and Refiner in both evaluated settings. Due to time constraints during the rebuttal period, we could not repeat all settings, baseline comparisons, and controlled ablations with DeepSeek-V4-Pro.

W4: Cross-Backbone and Cross-Benchmark Causal Replication

We thank the reviewer for requesting causal validation beyond Qwen–ALFWorld. We have added experiments on Llama-3.1-8B–ALFWorld and Qwen2.5-7B–ScienceWorld, replicating both core claims: “Utilization identifies when the same segment matters” and “Utilization identifies which memory segments matter.” The results appear in Figure 9, Appendix E.8 of the updated manuscript.

When the same segment matters. The two left panels repeat the same-segment temporal intervention in main-text Table 1. The maximum/minimum effect ratio is 3.01× for retrieval heads on Llama–ALFWorld, compared with 1.29× and 0.91× for the two random-head controls. On Qwen–ScienceWorld, the corresponding ratios are 7.03×, versus 1.03× and 1.26× for the controls.

Which memory segments matter. The two right panels repeat the fixed-decision segment intervention in main-text Figure 4. In both settings, the highest-utilization quintile produces much larger changes in recorded-response NLL than the other four quintiles.

These results reproduce the patterns observed on Qwen–ALFWorld, supporting both utilization claims across different backbones and environments.

Reviewer_ZpzGOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper argues that self-evolving agent memory should be updated using evidence about how retrieved memory was actually accessed during execution, not only from outcomes and reflection. The authors identify retrieval-sensitive attention heads per backbone and aggregate their attention into a segment-by-decision memory-use trace, which they validate with group ablations, temporal and segment-level interventions, and random-head controls. On top of this signal they build AGMR, which diagnoses one of three failure modes (accessed-but-ineffective, missed, distracted) and proposes a textual edit committed only after re-execution verification. On ALFWorld, ScienceWorld, and WebShop with Qwen2.5-7B and Llama-3.1-8B, AGMR outperforms Traj, AWM, and DC while generally shortening memory.

Critical strengths
  1. The read-side framing is a genuine contribution: prior memory systems update from what happened, whereas AGMR adds explicit evidence of how retrieved memory was accessed, with an intuitive three-case taxonomy that ties diagnoses to concrete edits.

  2. The validation of the memory-use trace is thorough, combining group ablations, temporal and segment-level interventions, random-head controls, bootstrap head selection, and held-out replications.

  3. The method is lightweight and practical: retrieval heads are selected once per backbone and reused across benchmarks, the trace reuses the existing decoding pass with no extra forward pass, and the verification-and-rollback mechanism makes every committed edit monotonically safe.

Critical weaknesses
  1. It is unclear how much of the gain comes from the attention signal itself. In Table 3, the text-only variant with the same Gemini-written edits and re-execution verification already improves Traj from 70.9 to 76.1 on Qwen-ALFWorld, leaving only about 5 points for the attention trace, a gap that may not be significant given about 134 test tasks and no reported variance. It is also unclear whether AWM and DC use the same verification, so AGMR's margin in Table 2 may largely come from verification and rollback.

  2. The baselines are limited. Only AWM and Dynamic Cheatsheet represent self-evolving memory, while more recent memory-update methods cited in the introduction, such as ReasoningBank and ACE, are not compared, which weakens the claim of best performance across all settings.

  3. The citation format is incorrect throughout the paper. Parenthetical citations are typeset as textual ones without parentheses (e.g., "long-horizon decision making Yao et al. (2022b); Park et al. (2023)".

Questions for authors

Null

2 comments

Response to Reviewer ZpzG

We sincerely thank the reviewer for the valuable comments, constructive feedback and for recognizing the contributions of our read-side memory utilization analysis and intervention-based validation. We have made additional experiments and revisions to address the concerns regarding the attribution of AGMR's improvements, baseline coverage, and citation formatting.

W1 and W2: Additional Baselines, Verification Protocol, and Variance Reporting

To address the concerns regarding baseline coverage, we have added ACE and ReasoningBank as additional baselines and expanded Table 2 accordingly.

Regarding the verification mechanism, DC, ACE, ReasoningBank, and AGMR all adopt the same verification-and-rollback protocol. Specifically, after updating the memory based on a task, we re-execute the original task using the updated memory. If the update hurts the original task performance, the update is rolled back and not retained. AWM is the only exception, as our implementation of AWM only summarizes successful trajectories offline and does not perform the same failure-driven memory update process.

Furthermore, to address the concern regarding the lack of variance reporting, we have added standard deviations to both Tables 2 and 3. The complete updated tables are provided below.

Updated Table 2. Performance comparison across environments and backbone models. For clarity, we report the results for Qwen2.5-7B-Instruct and Llama-3.1-8B-Instruct separately

Qwen2.5-7B-Instruct

EnvironmentMetricReActICLAWMDCTrajACEReasoningBankAGMR
ALFWorldPerformance ↑41.79 ± 045.52 ± 064.18 ± 0.4352.24 ± 0.7570.90 ± 059.70 ± 0.8652.24 ± 0.4381.34 ± 0.75
ALFWorldAvg. steps ↓28.40 ± 026.71 ± 022.22 ± 0.3525.09 ± 0.4619.36 ± 022.39 ± 0.3725.45 ± 0.0416.08 ± 0.15
ALFWorldMemory tokens ↓——312.01 ± 4.281013.73 ± 8.12852.36 ± 015929.00 ± 48.80145.97 ± 5.64752.53 ± 7.22
ScienceWorldPerformance ↑29.88 ± 029.38 ± 050.39 ± 1.3363.73 ± 0.4167.69 ± 044.27 ± 0.8030.80 ± 0.1877.20 ± 0.71
ScienceWorldAvg. steps ↓19.56 ± 018.35 ± 016.92 ± 0.1515.52 ± 0.0614.62 ± 018.34 ± 0.2719.14 ± 0.4313.78 ± 0.15
ScienceWorldMemory tokens ↓——435.32 ± 8.661478.85 ± 39.511040.62 ± 015974.64 ± 62.14158.03 ± 4.37990.89 ± 7.09
WebShopPerformance ↑56.66 ± 060.12 ± 056.11 ± 0.4454.04 ± 0.5951.04 ± 057.70 ± 1.0459.36 ± 0.8963.86 ± 0.57
WebShopMemory tokens ↓——185.65 ± 9.25618.14 ± 12.451372.22 ± 014042.51 ± 128.16180.75 ± 6.92503.77 ± 8.80

Llama-3.1-8B-Instruct

EnvironmentMetricReActICLAWMDCTrajACEReasoningBankAGMR
ALFWorldPerformance ↑0.00 ± 01.49 ± 07.46 ± 0.4311.19 ± 0.4326.12 ± 020.15 ± 1.7218.66 ± 0.8647.76 ± 0.86
ALFWorldAvg. steps ↓40.00 ± 039.74 ± 038.30 ± 0.2436.67 ± 0.0332.54 ± 034.66 ± 0.3735.23 ± 0.4925.52 ± 0.39
ALFWorldMemory tokens ↓——312.01 ± 4.281133.51 ± 17.07852.36 ± 015984.78 ± 213.29140.13 ± 6.18845.91 ± 5.82
ScienceWorldPerformance ↑26.50 ± 034.97 ± 052.14 ± 0.7672.28 ± 0.8375.63 ± 056.88 ± 0.2450.21 ± 0.9983.74 ± 0.45
ScienceWorldAvg. steps ↓19.59 ± 018.37 ± 016.23 ± 0.1213.99 ± 0.2513.22 ± 015.51 ± 0.3316.84 ± 0.2512.04 ± 0.11
ScienceWorldMemory tokens ↓——435.32 ± 9.961168.99 ± 22.591040.62 ± 014825.59 ± 83.79164.93 ± 8.58918.53 ± 19.90
WebShopPerformance ↑33.06 ± 052.10 ± 041.56 ± 1.5029.68 ± 1.6241.45 ± 049.29 ± 0.4357.86 ± 0.8363.26 ± 0.40
WebShopMemory tokens ↓——185.65 ± 7.82658.96 ± 25.111372.22 ± 014293.16 ± 58.72163.32 ± 7.90567.57 ± 34.99

Updated Table 3. Factor-matched attention controls.

BackboneRefinement settingAttention inputALFWorld: Performance / StepsScienceWorld: Performance / Steps
Qwen2.5-7B-InstructTwo-call text-onlyNone76.13 ± 0.43 / 17.36 ± 0.0472.85 ± 0.70 / 14.15 ± 0.17
Qwen2.5-7B-InstructRandom-5-headRandom heads75.37 ± 1.29 / 18.28 ± 0.1370.91 ± 1.28 / 14.38 ± 0.23
Qwen2.5-7B-InstructAGMRRetrieval heads81.34 ± 0.75 / 16.08 ± 0.1577.20 ± 0.71 / 13.78 ± 0.15
Llama-3.1-8B-InstructTwo-call text-onlyNone41.04 ± 0.75 / 28.06 ± 0.4779.72 ± 0.73 / 12.53 ± 0.13
Llama-3.1-8B-InstructRandom-5-headRandom heads38.81 ± 1.72 / 29.81 ± 0.2677.68 ± 0.21 / 12.79 ± 0.09
Llama-3.1-8B-InstructAGMRRetrieval heads47.76 ± 0.86 / 25.52 ± 0.3983.74 ± 0.45 / 12.04 ± 0.11

W3: Citation Formatting

We sincerely thank the reviewer for pointing out this issue. We have corrected the citation formatting throughout the paper to ensure that all citations follow the appropriate formatting conventions.

Reviewer_iBh5OpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes Attention-Guided Memory Refinement (AGMR), which extracts decision-level memory-use traces from retrieval-sensitive attention heads and combines them with semantic reflection to revise and compress persistent memory. The paper includes local attention interventions and budget-matched ablations. However, all end-to-end evaluations are restricted to Qwen2.5-7B and Llama3.1-8B, leaving the method’s relevance and incremental value for contemporary agents insufficiently established.

Critical strengths
  1. Budget-matched controls provide evidence for the incremental utility of the attention signal. In Table 3, AGMR improves over text-only refinement by approximately 4–6 points while retaining the two-stage pipeline, call budget, retry policy, and re-execution verification. This is the strongest positive result: it supports the signal’s utility within the evaluated configurations, although its scope remains limited by the backbone selection.

  2. The paper attempts interventional validation rather than relying solely on attention visualizations. Same-segment temporal interventions, fixed-decision segment interventions, and random-head controls support an association between high attention and local influence through the measured retrieval pathway. This provides some mechanistic grounding, although it does not establish the broader failure-diagnosis claims.

Critical weaknesses
  1. The evaluation relies exclusively on two dated, relatively limited small models and does not establish relevance to contemporary agents. For an ICLR 2027 submission, evaluation only on Qwen2.5-7B and Llama3.1-8B is insufficient. Notably, Llama3.1-8B achieves only 0% and 1.49% ALFWorld success with ReAct and fixed-demonstration ICL, respectively, indicating a substantial capability floor in part of the evaluation. The results therefore do not distinguish a broadly useful memory-refinement method from a technique that primarily compensates for these models’ weaker instruction following and context utilization. The absence of newer, stronger open-weight backbones with accessible attention is a major gap in assessing practical significance.

  2. The central mapping from attention patterns to failure causes and editing strategies remains insufficiently validated. The interventions establish that certain segments influence outputs through specified pathways, but do not directly establish that the three attention patterns accurately identify the claimed failure causes. Moreover, Appendix G localizes errors through deviations from a reference execution order, potentially misclassifying valid alternative trajectories. Consequently, the gains may arise primarily from improved segment localization and verification-based filtering rather than accurate mechanistic diagnosis. This limits the contribution beyond an empirical attention-assisted editing procedure.

Questions for authors
  1. Request for critical validation: Are there results on newer and more capable open-weight backbones? In particular, at a comparable parameter scale, does AGMR retain its advantage over budget-matched text-only refinement? This evidence would directly affect my assessment of whether the method generalizes beyond compensating for older models’ limitations.

  2. Clarification request: How does the Semantic Reflector distinguish genuine errors from valid execution paths that differ from the reference trajectory? Is there an independent audit quantifying error-step localization, supporting-segment localization, and the correctness of the three diagnostic categories?

8 comments

Response to Reviewer iBh5

We sincerely thank the reviewer for recognizing the evidence provided by our budget-matched controls and our efforts to validate the attention signal through interventions.

W1 and Q1: Evaluation on a Newer Backbone

We sincerely thank the reviewer for highlighting the importance of evaluating AGMR on newer and more capable open-weight backbones. To address this concern, we have extended our evaluation to Qwen3-8B.

We evaluated Qwen3-8B on ALFWorld and ScienceWorld, following the same experimental settings and baseline configurations as in Tables 2 and 3. Importantly, retrieval heads were selected using ALFWorld training cases and then transferred to ScienceWorld without any target-benchmark-specific head reselection.

The results are summarized below:

MethodALFWorld Success (%) ↑ALFWorld Memory Tokens ↓ScienceWorld Avg. Reward (×100) ↑ScienceWorld Memory Tokens ↓
ReAct21.64—43.94—
ICL33.58—65.96—
Traj54.48872.0791.821021.73
AWM44.78476.9064.74488.77
DC76.87791.2584.501023.39
Two-call text-only refinement74.63753.7588.27793.97
Random-5-heads69.40770.2990.49830.11
AGMR83.58727.9993.04813.44

ALFWorld. On Qwen3-8B, AGMR's attention guided refinement is surprisingly effective, achieving an 83.58% success rate, a substantial improvement of 29.10 percentage points over the original, unmodified trajectory memory (Traj). It also outperforms Dynamic Cheatsheet by 6.71 percentage points and budget-matched two-call text-only refinement by 8.95 percentage points. Notably, Qwen3-8B achieves substantially higher ReAct and ICL success rates (21.64% and 33.58%, respectively) than the near-zero baselines observed for Llama-3.1-8B in the original evaluation. The consistent improvement over the text-only control therefore provides additional evidence that AGMR's benefits are not limited to the original weaker-backbone setting.

ScienceWorld. The improvement on ScienceWorld is relatively limited, primarily because the Traj baseline already achieves a very high average reward of 91.82, leaving limited headroom for further improvement. Nevertheless, AGMR achieves the highest average reward of 93.04. Moreover, compared with Traj, AGMR reduces the average injected memory from 1,021.73 to 813.44 tokens (20.39%), demonstrating improved memory efficiency alongside the performance gain.

Overall, these additional results extend AGMR's evaluation to a newer backbone and demonstrate that its advantage over budget-matched text-only refinement persists across both ALFWorld and ScienceWorld. We will incorporate these results into Tables 2 and 3.

W2 and Q2: Reliability of Attention-guided Failure Diagnosis

Clarification of the three failure categories. Our experiments provide evidence that these attention patterns can guide effective, targeted memory edits. In Figure 5, controlled interventions produce the expected behavioral changes for different patterns. For missed-memory cases, boosting the intended segment improves the correct-versus-incorrect response preference; for distraction cases, suppressing competing memory produces the expected improvement. Furthermore, Appendix F (Figures 12 and 13) provides concrete case studies illustrating how these patterns guide corresponding memory edits and how the resulting changes improve memory utilization and agent behavior. Thus, our claim is that attention patterns provide actionable signals for memory refinement, rather than that they constitute an exhaustive explanation of the underlying causes of all agent failures.

Clarification of error-step localization. That's a prompt construction question. We agree that deviation from a single reference trajectory does not necessarily imply an execution error, since multiple valid action sequences may achieve the same task objective. What we actually do with the Semantic Reflector is use the successful Expert Trajectory to distill a canonical execution workflow, which serves as a verified reference for task completion. This is similar to supervised fine-tuning (SFT), where a successful demonstration is used as a concrete optimization target. We learn from and optimize toward this demonstrated workflow, rather than attempting to enumerate or learn all possible valid execution paths. Choosing one successful trajectory as a reference does not imply that alternative valid paths are incorrect. Instead, deviations from this canonical workflow indicate at least some potential opportunities to further optimize the execution toward this nearly optimal workflow.

Independent diagnostic audit. To directly address the reviewer's concern, we conduct a blinded human audit on 200 failure cases across three benchmarks. Two independent annotators evaluate the trajectories without access to the Semantic Reflector's original predictions or AGMR's diagnostic labels. Specifically, they independently annotate (1) the first genuinely erroneous decision; (2) the supporting memory segment for the appropriate decision; and (3) the corresponding attention-based memory-access pattern, based on the observed memory utilization traces.

Disagreements are resolved through discussion to establish the final human-annotated references. We evaluate three aspects of AGMR's diagnostic pipeline:

  • Error-step localization: Semantic Reflector achieves 92.0% exact-match accuracy in identifying the first erroneous decision.
  • Supporting-segment localization: Semantic Reflector achieves 89.5% exact-match accuracy in identifying the supporting memory segment.
  • Three-category diagnosis: AGMR achieves 89.0% accuracy in distinguishing the three memory-access patterns against human annotations.

These results provide independent quantitative evidence for the reliability of AGMR's localization and diagnosis pipeline.

Thank you for adding the Qwen3-8B experiments. I appreciate that AGMR improves over the budget-matched text-only control on both benchmarks.

One result I would like clarified is the size of the ALFWorld baseline drop from Qwen2.5-7B-Instruct to Qwen3-8B. ReAct falls from 41.79% to 21.64%, and Traj from 70.90% to 54.48%. Qwen3-8B performs better on several public benchmarks, and other independent ALFWorld evaluations usually report advantage for Qwen3-8B. Could this discrepancy indicate a mismatch between the two evaluation setups, particularly in thinking mode, generation budget, or action parsing? And my main concern remains unresolved because the audit does not report how often the Reflector correctly identifies the first genuine error in trajectories where the agent previously took a valid action that deviated from the reference trajectory.

For now, I am keeping my rating at 2.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response

Dear Reviewers and Area Chair,

Thank you all for your thoughtful feedback and for engaging with us during the discussion. We have uploaded a revised manuscript incorporating the main changes discussed in our responses. The revisions are highlighted in blue.

  • We added ACE and ReasoningBank to Table 2 and reported standard deviations in Tables 2 and 3.
  • We added Qwen3-8B results on ALFWorld and ScienceWorld in Appendix E.12.
  • We added retrieval-head selection robustness experiments using a different annotation model and new supervision sets in Appendix D.2.1.
  • We added causal intervention replications on Llama–ALFWorld and Qwen–ScienceWorld in Appendix E.8.
  • We expanded the related work discussion to cover Self-RAG and corrected the citation formatting.

We appreciate your time and consideration.

Best regards,
The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.