acceptodds
Under review as a conference paper at ICLR 2027

SAM: State-Adaptive Memory for Long-Horizon Reasoning Agent

Abstract

Long-horizon agentic reasoning requires large language models to act over long interaction histories containing thoughts, tool calls, observations, and partial conclusions. The challenge is not merely that these histories grow long, but that information needed for the current decision may be scattered across distant steps and only become relevant later. Existing approaches address this difficulty by truncating the interaction history, compressing it into shorter surrogates, or retrieving selected parts of it for reuse, but they do not explicitly model how access to past interaction should adapt to the agent's evolving state. We instead cast long-horizon reasoning as a problem of state-adaptive memory. To this end, we propose State-Adaptive Memory (SAM), a standalone framework that consolidates ongoing interaction into compact memory cues while preserving raw trajectory pages for intent-driven recall. These cues are not treated as replacements for history; rather, they serve as lightweight handles that allow the agent to reconstruct temporally distant information according to its current needs, without retraining the underlying backbone. We further optimize the memory module through expert-guided supervision and reinforcement learning, aligning it with trajectory-level utility. Across BrowseComp, BrowseComp-ZH, WideSearch, and HLE, SAM consistently outperforms strong baselines over diverse agent backbones. Our results suggest that explicit memory modeling provides a simple and effective foundation for long-horizon agentic reasoning.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Memory, Context Management

Reviews

Among:
Shape:
1234this paper, Top 17%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.75 over 4 reviews, above 83% of the others.
Reviewer_gGG4OpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes State-Adaptive Memory (SAM), an external memory framework for LLM agents performing long tasks. Its central idea is to keep past interactions recoverable without carrying the entire history in the agent’s context. SAM divides interaction histories into pages, stores the original pages externally, and retains compact summaries called memory cues in the active context. When earlier information becomes relevant, the agent selects cues and specifies what it needs to recall. A separate memory model then reads the corresponding pages and returns information tailored to that request.

The authors train a Qwen3.5-9B memory model while keeping the main agent frozen. Training combines supervised learning from successful expert trajectories with Oracle-Anchored Tree GRPO (OAT-GRPO). This reinforcement-learning method branches at memory calls to compare alternative recall outputs, using both downstream task success and assessments against reference outputs from stronger models.

Experiments cover BrowseComp, BrowseComp-ZH, WideSearch, and HLE with two agent backbones. SAM improves the average benchmark score from 49.4 to 57.0 for GLM-4.7 and from 44.5 to 48.8 for Qwen3.5-35B-A3B, outperforming the tested truncation and summarization baselines. Ablations support the usefulness of both training stages and intent-conditioned recall. The main contribution is a reusable memory module that combines compact context with on-demand access to detailed history, allowing memory access to change as the agent’s task progresses.

Critical strengths
  1. SAM offers a practical way to preserve access to detailed history while keeping the agent’s active context compact. Its memory cues point to stored trajectory pages, allowing the agent to recover information according to its current needs. The recall ablations support this design: on BrowseComp, full SAM achieves 42.2, compared with 39.0 for summary-only memory and 38.0 for raw-page readout. This is meaningful evidence that selective reconstruction can help beyond simply retaining or summarizing history.

  2. The same trained memory module improves results across two frozen agent backbones and four benchmarks. SAM outperforms the included context-management heuristics in all eight backbone–benchmark combinations. Although this does not establish superiority over the closest learned memory systems, the consistency is encouraging. It supports the practical value of a reusable memory component that can improve different agents without retraining their reasoning models.

Critical weaknesses
  1. The contribution over closely related memory and training methods is not sufficiently established. GAM[1] already combines lightweight memory with a complete page store for request-conditioned retrieval, while Mem-T[2] uses memory-operation trees to derive intermediate training signals from downstream outcomes. Neither is discussed. The controlled comparisons are primarily against truncation and summarization heuristics, leaving unclear what SAM’s architecture and OAT-GRPO add beyond existing approaches.

  2. The experiments do not adequately isolate the proposed mechanisms, and a configuration mismatch makes the gains harder to assess. Removing OAT-GRPO shows that the complete RL stage helps, but does not distinguish the effects of tree-based credit assignment, task rewards, and committee-based rewards. There is also no untuned memory model using the identical SAM architecture. Together, these gaps leave uncertainty about which components produce the gains and how consistently those gains are measured.

[1] Yan, B. Y., et al. "General agentic memory via deep research." arXiv preprint arXiv:2511.18423 (2025).

[2] Yue, Yanwei, et al. "Mem-t: Densifying rewards for long-horizon memory agents." arXiv preprint arXiv:2601.23014 (2026).

Questions for authors
  1. What distinguishes SAM’s cue–page architecture from GAM’s lightweight memory and complete page store, and OAT-GRPO from Mem-T’s memory-operation tree training? Which differences do you consider essential to the reported improvements?

  2. Do you have results separating tree-based credit assignment from the committee and task-outcome rewards, ideally with matched training budgets? An untuned 9B model using the same SAM architecture would also help establish how much improvement comes from the architecture versus training.

8 comments
AuthorsOpenReview ↗

[Part 1/4] Response to Reviewer gGG4

Thank you for recognizing the practical value of SAM in preserving detailed history while keeping the active context compact, and for noting the evidence from read ablations and consistent gains on two frozen backbones. To address your concerns about related work, the contributions of the mechanisms, and configuration consistency, we first clarify the scope of this paper, then explain the roles of the architecture and training separately.

First, SAM studies working memory within a single long-horizon task, which can also be understood more directly as context management. Memory in this paper does not span sessions: pages and cues support the ongoing execution of the current task and are not passed to other tasks as a cross-session memory store. We use the term Memory following the functional taxonomy in Memory in the Age of AI Agents. In §4.3, that survey defines working memory as mechanisms for actively managing and manipulating context within a single episode, and includes state maintenance across multiple interaction turns in this category. Thus, SAM's memory focuses on preserving, selecting, and reconstructing historical information within the current task.

This positioning also defines the main problem of the paper: how a frozen main agent can continue using earlier information as search, tool feedback, and intermediate judgments accumulate, when the importance of that information only becomes clear later. Connecting the trained memory module to different main agents means reusing its memory-processing capability; the actual history stored for each task remains separate.

W1 / Q1: Positioning

You noted that GAM already has lightweight memory and a complete page store, and that Mem-T already uses memory-operation trees to assign intermediate feedback, so we need to explain what SAM adds. Our distinction is to embed memory processing in the execution of a single task and train an external module to reconstruct the historical evidence currently needed into content usable for the next decision. Below, we explain the relationship to each work and which experiments directly test this design.

1. Comparison with GAM: both preserve recoverable history, but the access workflows and evaluation goals differ.

GAM uses a Memorizer to maintain lightweight memory and a page store, and a Researcher to iteratively plan, search, synthesize, and reflect around a request. It also discusses a framework that freezes the client and optimizes the memory module. Thus, SAM's differences need to be specified in terms of online history processing and recall training; they cannot be summarized solely as a structure of summaries plus raw text.

In SAM, raw pages come from the task the main agent is currently executing. Once the context reaches the threshold, the system consolidates history and leaves visible cues. The main agent then selects pages and issues a recall intent based on its current judgment. The memory model reconstructs the needed information directly from these pages, and the main agent continues searching or reasoning after receiving it. For example, the same history may help identify candidate entities early on and later be needed to check a time constraint. Information needs change with the decision state, and an early summary may not cover later needs in advance.

Thus, SAM's core test is whether reconstruction actually improves later execution when the same history is reused in different states. GAM's request-conditioned synthesis overlaps with this goal. SAM further trains recall outputs within an ongoing task and evaluates their utility through actual task outcomes.

AuthorsOpenReview ↗

[Part 2/4] Response to Reviewer gGG4

2. Comparison with Mem-T: the focus is the current task's working context, rather than the full lifecycle of a memory store.

Mem-T organizes working, factual, experiential, and raw memory into a hierarchical system and learns operations such as formation, updating, and retrieval. It includes working memory, so the distinction is not whether working memory is present. Rather, SAM focuses on history management and recall reconstruction within a single task, while Mem-T also addresses broader memory-store maintenance.

DimensionGAMMem-TSAM
Managed contentContinuously growing history and its page storeWorking, factual, experiential, and raw memoryReasoning and interaction history in the current session
Main access operationsIterative search, synthesis, and reflection around a requestRetrieval from different memory modules, combined with memory formation and updatesThe main agent specifies intent and pages; the external model reconstructs the needed content
Use goal considered hereGenerate context for client requestsMaintain and use a hierarchical memory storeReuse earlier evidence from the current task for later decisions

In training, Mem-T's tree nodes represent memory retrieval operations, and retrieval utility is traced back to memory construction. SAM branches at the same recall request to compare different readouts under the same main-agent state. Mem-T uses intra-tree and inter-tree advantage estimation, while SAM forms comparison groups from sibling candidates sharing a parent state. Their auxiliary feedback also focuses on retrieved evidence and recall recoverability based on raw pages, respectively.

The reason for this design is that we want to train the memory content returned to the main agent at this particular call. Comparing candidates under the same need and page conditions reduces the effect of differences in task state on the training signal. The tree structure serves the memory mechanism here, and its separate role is tested in the Optimization ablations below.

3. Read ablations directly test the value of intent-conditioned reconstruction.

You have already noted the difference between Raw-content and full SAM. Here, we further link each baseline to the specific mechanism it tests: Figure 2 fixes the write side and page store and compares how the same history is used.

Read methodQwen BrowseCompFactor tested
Summary-only39.0%Uses only the summary formed at write time
Recency37.0%Accesses pages by recency, without guidance from current needs
Raw-content38.0%Provides raw pages without intent-conditioned reconstruction
SAM42.2%Reconstructs relevant information from raw pages based on current intent

SAM improves over Raw-content by 4.2 percentage points and over Summary-only by 3.2 percentage points. The former shows that even after raw text is saved and made available again, how this information is processed still affects task outcomes. The latter shows that write-time summaries cannot fully replace recovering details for new needs at read time. Together, the two comparisons support preserving recoverable history and reorganizing information when needed.

Adapting GAM-style retrieval, synthesis, and reflection to the same online task interface, with the same Qwen main backbone, tools, and system-wide total token cap, gives 41.0% on BrowseComp, compared with 42.2% for full SAM. This comparison shows that SAM's complete access and training scheme provides additional gains under the current task protocol. The difference includes the joint effects of workflow and training. The same-architecture comparison below further separates the contribution of training, avoiding attributing both effects entirely to the page interface.

AuthorsOpenReview ↗

[Part 3/4] Response to Reviewer gGG4

W2 / Q2: Architecture

You suggested that an untuned 9B model under the same architecture would help separate the contributions of architecture and training. We address this by keeping storage, the call interface, and the main agent unchanged, and varying only the memory model's training status. This directly tests whether gains come from connecting this interface or also depend on the memory model learning to use it more effectively.

ConfigurationGLM average over four tasks
No context management49.4
Same SAM architecture, untuned 9B54.3
Same SAM architecture, SFT-only55.6
Same SAM architecture, w/o SFT, RL-only55.9
Same SAM architecture, SFT + OAT-GRPO57.0

1. Connecting an untuned module already helps, but specialized training adds further gains.

The untuned 9B model reaches 54.3 when connected to SAM, improving by 4.9 points over no context management. This shows the practical value of the full interface for history consolidation, storage, and on-demand access. With exactly the same interface, SFT raises the score to 55.6, and full training further raises it to 57.0. The full model's 2.7-point improvement over the untuned model therefore cannot be explained solely by adding another model or keeping raw logs.

This also explains why SAM studies both the architecture and external-module training: the interface makes history usable, while training improves how much the returned content helps the current task. Both contribute to the final result.

2. The paper's w/o SFT variant and a fully untuned model are different controls.

The w/o SFT score of 55.9 still includes RL, so it tests whether reinforcement learning needs supervised initialization. The newly added untuned 9B model receives neither SAM SFT nor RL, and tests what the interface can achieve without specialized training. Presenting them side by side keeps gains from training from being counted as gains from the untrained architecture.

The main agent's parameters remain frozen throughout. RL uses GPT-5.4 for subsequent reasoning, and gradients update only the memory model. After training, the same module is connected to GLM and Qwen. These differences therefore reflect changes in external memory-processing capabilities.

W2 / Q2: Optimization

Your concern is how much tree-local credit assignment, task outcomes, and committee feedback each contribute, rather than just whether removing all RL lowers the score. We cross the credit-assignment methods with reward components under the same architecture, SFT initialization, and training tasks, matching rollout token budgets to test local comparison and auxiliary feedback separately. The budget includes subsequent execution by the frozen reasoner. Mixed-reward groups use the same committee configuration and scoring method.

MethodReward componentsGLM average over four tasks
SFT-only—55.6
Standard trajectory-level GRPOTask outcome only55.9
Standard trajectory-level GRPOTask outcome + recoverability56.2
Tree-local credit assignmentTask outcome only56.4
Tree-local credit assignmentRecoverability only55.8
Full OAT-GRPOTask outcome + recoverability57.0

1. Local credit assignment links task feedback to specific recall actions.

Final failure of a long trajectory does not mean that every recall in it had no value. For example, the memory model may have recovered key time information, but the main agent may make an incorrect judgment later. If all intermediate actions share one final outcome, it is difficult to distinguish which recalls are worth retaining.

OAT-GRPO generates different readouts with the same intent, pages, and main-agent state, then lets the frozen reasoner continue execution. The comparison thus starts from the same point, and the training signal is more directly tied to the effect of the current readout. With task outcome rewards alone, local credit assignment improves over trajectory-level GRPO by 0.5 points. When both use mixed rewards, it still improves by 0.8 points. These gains therefore remain under the same reward conditions, supporting comparisons at the level of memory actions themselves.

Reviewer_7bRaOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces SAM, an external memory framework for long-horizon reasoning agents. When the agent needs to revisit previous information, it can specify the relevant IDs pages and what to recall. A separate memory model then reconstructs the needed information. The memory model is initially trained using expert supervision and further optimized with Oracle-Anchored Tree GRPO. The authors evaluate SAM on BrowseComp, BrowseComp-ZH, WideSearch, and HLE with improvements on two different agent backbones.

Critical strengths
  1. Separating compact memory from the original history is a useful design. Information omitted during consolidation can still be retrieved when needed, and the agent can decide which pages to revisit based on its current task. This is useful for long-horizon tasks.

  2. The experimental results are consistent. The comparison with raw-content recall is informative, suggesting the gains from how information is retrieved rather than simply preserving the original history.

Critical weaknesses
  1. The training ablation does not clearly establish the contribution of OAT-GRPO. While the paper emphasizes tree-local credit assignment and the committee-based recoverability reward, the ablation only removes the entire RL stage.

  2. Some improvements over the strongest baseline are relatively small, such as 0.7 points on HLE and 0.9 points on WideSearch with GLM-4.7. BrowseComp and HLE also use only 200 sampled questions.

  3. SAM requires a separate memory model for consolidation and recall, but the paper does not report the total token consumption or latency.

Questions for authors
  1. Can the contributions of tree-local credit assignment and the committee-based reward be evaluated separately? Also, which frozen reasoning backbone was used during RL training of the shared memory model?

  2. Can the authors report total token usage and model-call counts, including the memory model, for SAM ?

  3. Can the authors clarify the difference between Table 1 and Figure 3(c)? Appendix B.1 specifies pages of up to 32K tokens, but the 32K setting in Figure 3(c) gives 44.0/48.4 on BrowseComp/BrowseComp-ZH with Qwen, compared with 42.2/46.5 in Table 1.

6 comments
AuthorsOpenReview ↗

[Part 1/4] Response to Reviewer 7bRa

Thank you for recognizing the value of separating compact memory from raw history, the consistency of experiments across backbones, and the support that the raw-content baseline provides for the read mechanism. We address your questions about the contribution of training, evaluation scale, inference cost, and configuration differences below.

W1 / Q1: Optimization

You noted that removing the entire RL stage only shows that RL helps overall, but does not separate the contributions of tree-local credit assignment and committee rewards. Our explanation is that these two designs address different training difficulties: local credit assignment compares which recall is more useful in the same state, while committee feedback adds information about readout quality that final success or failure cannot distinguish. To address this attribution issue, we fix the memory architecture, SFT initialization, and training tasks, and vary the credit-assignment method and reward components under comparable rollout token budgets.

MethodReward componentsGLM average over four tasks
SFT-only—55.6
Standard trajectory-level GRPOOutcome only55.9
Standard trajectory-level GRPOOutcome + recoverability56.2
Tree-local credit assignmentOutcome only56.4
Tree-local credit assignmentRecoverability only55.8
Full OAT-GRPOOutcome + recoverability57.0

1. Local credit assignment ties training comparisons to specific memory actions.

A long trajectory may contain multiple recalls, and final failure does not directly mean that every recall was ineffective. For example, a recall may have recovered the necessary evidence, but the main agent may still make an incorrect judgment later. With only one outcome for the entire trajectory, training has difficulty distinguishing the value of these intermediate actions.

OAT-GRPO generates candidate readouts with the same intent, pages, and main-agent state, then compares their utility through the outcomes of their later branches. The local baseline thus compares which readout is more helpful for the same information need, reducing the effect of differences between trajectory states on memory-quality assessment.

The corresponding evidence in the table is that local credit assignment exceeds trajectory-level GRPO by 0.5 points with outcome rewards alone, and by 0.8 points when both use mixed rewards. Thus, these gains cannot be explained solely by adding committee supervision; the credit-assignment structure itself adds value.

2. Committee feedback provides local quality information beyond final outcomes.

The same recall request can have multiple valid outputs. At the same time, two readouts of different quality may receive the same task outcome because of later reasoning. The committee provides complementary references for the same intent and raw pages, allowing the model to receive signals about relevance, evidence coverage, and faithfulness even when final feedback cannot distinguish candidates.

With the tree structure fixed, adding recoverability feedback raises the average from 56.4 to 57.0, showing that this signal supplements the outcome reward. In contrast, recoverability alone gives 55.8, clearly below the full method. This shows that reference-model evaluation cannot replace actual task outcomes either. Both constraints are needed: whether the current readout is sufficient, and whether it ultimately helps the task succeed.

Thus, compared with removing the entire RL stage, these crossed ablations directly address your concern: the full gains include both the contribution of local memory-action credit assignment and the added value of recoverability feedback.

3. RL training uses frozen GPT-5.4, and only the memory model is updated.

We chose GPT-5.4 as the training reasoner to reduce the interference of the main agent's own reasoning errors in memory evaluation, so that useful recalls have a better chance to show their value in later execution. Training updates only the Qwen3.5-9B memory model, not GPT-5.4. Training follows Appendix B.4, with a branching factor of 3, a maximum depth of 3, and 200 rollout iterations. The comparisons above also control the actual rollout token budget, so that a larger sampling cost is not mistaken for an effect of the credit-assignment method.

AuthorsOpenReview ↗

[Part 2/4] Response to Reviewer 7bRa

W2: Evaluation

You noted that GLM's gains on HLE and WideSearch are small, and that BrowseComp and HLE use only 200 sampled questions. We assess the method's effectiveness through controlled comparisons with repeated sampling, consistent trends across two backbones, and read-mechanism ablations with history storage fixed. We first explain the existing avg@3 setting and sample selection, then explain how these results jointly support the conclusion.

1. The main table already uses avg@3 to reduce the effect of randomness in a single execution.

Each question is run independently three times under the same configuration, and the results are averaged. Thus, the main table is not based on a single run per question. In long-horizon tasks, a chance choice of search path can change the final outcome. Repeated execution reduces such run-to-run variation and makes comparisons more stable. The methods also use the same question sets, backbones, and tools, so that differences in question composition or main-model capabilities are not counted as gains from memory methods.

2. The 200-question setting reflects the cost of long-horizon evaluation, while other tasks test the full datasets.

Each BrowseComp and HLE run involves ongoing reasoning and tool interactions. With avg@3, each method configuration requires 600 complete rollouts on each 200-question task. Across two backbones, multiple baselines, and training ablations, the total evaluation cost is substantial. We therefore use fixed sampled question sets to retain repeated runs and multiple controlled comparisons within the available resources.

Similar work also provides precedents for this choice. In addition to ParallelMuse randomly selecting 200 BrowseComp questions, AgentSwing, cited in this paper, also uses a random 200-question BrowseComp subset in its main evaluation. IterResearch, a baseline in Table 1, uses a 200-question BrowseComp subset in its interaction-scaling experiments to compare performance under different interaction budgets. These settings show clear precedents for using fixed-size subsets in method comparisons and controlled analyses of computationally expensive long-horizon agents.

These precedents explain the practical basis for subset evaluation. Evidence for our method's gains comes from our own controlled comparisons: context-management methods run on the same fixed question sets, use avg@3 to reduce execution variation, and are also tested on the full BrowseComp-ZH and WideSearch datasets. Our evaluation thus combines comparisons on the same questions, repeated execution, and tests across datasets. Below, we explain how these results jointly support SAM's effectiveness.

3. The two small gains are not the only evidence supporting the method.

Each comparison below uses the strongest context-management heuristic for the same backbone and task, rather than a single, weaker baseline for all settings. The values come directly from Table 1:

TaskGLM: SAM gainQwen: SAM gain
BrowseComp+3.0+1.7
BrowseComp-ZH+1.7+1.5
HLE+0.7+1.2
WideSearch+0.9+2.3

The unit is percentage points. The two GLM gains you pointed out are indeed small, but the gains remain positive on the same tasks with Qwen, including 2.3 percentage points on WideSearch. Both backbones also improve on BrowseComp and BrowseComp-ZH. We therefore emphasize applicability across settings, rather than base the entire conclusion on one difference of 0.7 or 0.9.

More direct evidence for the mechanism comes from Figure 2: with the raw pages and write side fixed, Raw-content scores 38.0%, Summary-only 39.0%, and full SAM 42.2% on Qwen BrowseComp. This compares different uses of the same history and thus supports the value of intent-conditioned reconstruction beyond preserving history. The consistent trends in the main table and this controlled difference jointly support our conclusion.

AuthorsOpenReview ↗

[Part 3/4] Response to Reviewer 7bRa

W3 / Q2: Inference Cost

You pointed out that a separate memory model adds calls, and that reporting only the same context window and round cap does not reflect the actual cost. We report costs for the complete system: SAM trades extra computation for history consolidation and reconstruction for task gains. Its practical value must be assessed through accuracy, total usage, and latency together.

1. The statistics cover all inference roles, not only the main agent.

The following are per-question averages on BrowseComp. Total tokens include inputs and outputs from the task model, memory model, and auxiliary model. Model calls include consolidation and recall, and end-to-end time covers the full execution from task start to final answer. The teacher committee is used only during training and does not take part in these online calls.

Backbone / methodAccuracyTotal tokensTotal model callsEnd-to-end time
Qwen, discard-tool40.5%0.74M29.6405 seconds
Qwen, Summary39.5%0.78M33.4430 seconds
Qwen, SAM42.2%0.88M37.8470 seconds
GLM, Summary53.5%0.92M35.7510 seconds
GLM, SAM56.5%1.02M40.1550 seconds

2. Comparison with the strongest baseline makes the extra cost of the gains explicit.

For Qwen, SAM improves over discard-tool by 1.7 percentage points, with about 18.9% more total tokens and an increase in average time from 405 to 470 seconds. For GLM, SAM improves over Summary by 3.0 percentage points, with about 10.9% more total tokens and an increase in time from 510 to 550 seconds.

The two baselines also incur costs in different ways: discard-tool directly drops history and requires little extra processing, while Summary already calls a model to organize history. Compared with Summary, Qwen with SAM improves by 2.7 percentage points with about 12.8% more total tokens. Costs should therefore be understood relative to the specific alternative; not all context-management methods have zero overhead.

3. The extra memory computation provides recoverable history and reconstruction for current needs.

SAM's extra computation keeps early details usable after they leave the active context. When new clues change the current judgment, the agent can reread the original evidence without being limited by omissions in earlier summaries. The totals above already include the calls needed for this capability, so the accuracy gains do not come from leaving memory-service costs uncounted.

These results show a clear accuracy–cost trade-off. Simply dropping history still has an advantage when lower latency or fewer calls is the main goal. When tasks require recovering evidence across distant steps, SAM achieves a higher completion rate while incurring the extra costs above. This is the basis for our assessment of the separate memory module's practical benefits.

Reviewer_9KiSOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes State-Adaptive Memory (SAM), a memory framework for long-horizon LLM agents. The method maintains compact memory cues while preserving raw interaction pages. A Qwen3.5-9B memory model is first trained using GPT-5.4-generated demonstrations, then optimized through a proposed Oracle-Anchored Tree GRPO algorithm. Experiments on BrowseComp, BrowseComp-ZH, WideSearch, and HLE demonstrate consistent performance improvements over several baselines across two frozen agent backbones.

Critical strengths
  1. Practical and modular memory framework. SAM combines compact memory cues with preserved raw interaction histories, enabling intent-driven retrieval of relevant information without modifying the underlying agent backbone. This design addresses information loss in conventional context compression while remaining applicable to different reasoning models.

  2. Consistent empirical improvements. SAM achieves improvements across four long-horizon reasoning benchmarks and two agent backbones. The ablation studies further demonstrate the benefits of intent-driven recall over summary-only and raw-content retrieval strategies, providing empirical support for the proposed memory mechanism.

  3. Task-oriented memory optimization. The proposed OAT-GRPO framework incorporates tree-structured outcome rewards and oracle-guided feedback to optimize memory outputs. This approach is meaningful because it attempts to align memory quality with downstream task performance rather than relying solely on the quality of generated summaries.

Critical weaknesses
  1. Limited methodological novelty. The core memory architecture, which combines compact memory cues with preserved raw histories, closely resembles existing approaches such as ReadAgent. Similarly, intent-conditioned retrieval and tree-structured credit assignment have been explored in prior work. The paper does not sufficiently clarify its technical distinctions from these methods or establish the novelty of their integration, making it difficult to assess the significance of its methodological contributions.

  2. Insufficient experimental validation of the proposed optimization method. Although OAT-GRPO improves performance, its contribution is not adequately isolated through ablation studies. In particular, comparisons with standard GRPO and separate evaluations of outcome-based and oracle-based rewards are missing. Furthermore, the substantial reliance on frontier LLMs for both expert supervision and reward construction raises questions about whether the memory model learns task-optimized behavior beyond teacher imitation. These limitations weaken the empirical support for the proposed training strategy.

Questions for authors
  1. Novelty: What are the key technical differences between SAM and existing memory frameworks such as ReadAgent, and between OAT-GRPO and prior tree-structured GRPO methods?

  2. OAT-GRPO ablation: How does OAT-GRPO compare with standard GRPO? What are the individual contributions of tree-structured credit assignment and oracle-anchored rewards?

  3. Expert supervision: How is the reliability of GPT-5.4-generated memory traces validated, and how do the authors ensure that the learned memory model improves task utility beyond simply imitating the teacher?

7 comments
AuthorsOpenReview ↗

[Part 1/4] Response to Reviewer 9KiS

Thank you for recognizing SAM's modular design, its empirical performance across backbones, and the direction of optimizing memory behavior through task feedback. To address your questions about positioning and the contribution of training, we first explain the dynamic memory problem studied in this paper and its differences from ReadAgent. We then discuss the mechanisms and experiments of OAT-GRPO, followed by the reliability of expert supervision.

W1 / Q1: Positioning

You noted that separating compact memory from raw history and using tree-structured training both have related prior work. This paper mainly focuses on a memory mechanism for long-horizon agents: managing continuously growing interaction histories and recovering information based on current needs, while keeping the main agent's parameters unchanged. OAT-GRPO serves to optimize this memory mechanism. We first explain the differences in the problem and design below. The detailed tree-method comparisons and main experiments are presented in W2/Q2.

1. ReadAgent mainly handles long inputs given in advance, while SAM handles history that keeps growing during execution.

ReadAgent's main workflow divides a given long document into pages, generates gists, and then looks back at the original text based on a question. Its main experiments are on QuALITY, NarrativeQA, and QMSum. Thus, the long context mainly comes from input that already exists when the task begins, rather than new interactions produced as the agent solves the task.

SAM faces a different situation: the complete history does not yet exist when the task begins. The main agent keeps searching, reading tool results, trying candidate directions, and revising its judgments, while the history grows. An early detail may seem unimportant at the time but become key evidence after new constraints appear. Memory management therefore needs to run throughout execution and reinterpret earlier records once the current need becomes clear.

DimensionReadAgent's main settingSAM
Source of long contextLong documents given in advanceReasoning, actions, and tool feedback accumulated online
When memory is processedLook back based on the question after paging and gistingOngoing consolidation during execution and recall based on current intent
Use of detailed informationSupport understanding of and answers about the given materialSupport later search, verification, and decisions

2. In agentic settings, providing a memory tool does not mean the main agent will actively use it.

In practice, we observed that a main agent not trained to use memory tools may not call them actively or sufficiently, even when given a memory interface. This leaves a practical gap between preserving history and providing read tools, and using history effectively during long-horizon execution.

Our design aims to connect memory capabilities to existing main agents without retraining their reasoning and tool-use policies just to introduce memory. SAM therefore places learnable consolidation and reconstruction in a separate memory module. Once the context threshold is reached, the system triggers consolidation and retains visible cues. When detailed history is needed, the main agent only needs to select pages and state its current intent; the memory module processes the raw records.

This division of work means that organizing history does not depend on the main agent actively initiating writes, nor does it require the main agent to carry out the full reconstruction of a long history itself. Recall requests still come from the main agent. Freezing the main agent keeps its parameters and existing capabilities unchanged during memory training; it does not mean that training the external module alone removes all cases of insufficient tool use.

3. The paper centers on the memory mechanism, while the tree structure optimizes the downstream utility of memory actions.

SAM learns cue generation and intent-conditioned reconstruction. The main agent continues solving the task, while the memory module turns past information into content that can be used now. Even with the same raw pages, different reconstructions may lead to different later decisions.

OAT-GRPO therefore branches at memory calls and compares different readouts from the same state. The focus here is the link between what is optimized and the memory interface. Its specific differences from existing tree-based methods, along with the experimental contributions of tree-local credit assignment, reward components, and the memory mechanism, are presented together in W2/Q2 below.

AuthorsOpenReview ↗

[Part 2/4] Response to Reviewer 9KiS

W2 / Q2: Optimization

You suggested that comparisons with standard GRPO and separate evaluations of outcome and reference-model rewards are needed to establish the contribution of OAT-GRPO. Our response has two parts: first, clarify which actions the tree branches on and how feedback is assigned; then separate these factors under a fixed architecture and training budget.

1. Branches vary the memory outputs, and updates apply to the separate memory model.

At a recall request, the current intent, selected pages, and main-agent state are held fixed, while the memory model generates different candidate readouts. Each candidate is passed to the frozen reasoner, which continues execution, and later memory calls create further branches. Sibling nodes thus compare the effects of different memory outputs for the same need, rather than directly comparing entire reasoning trajectories with different initial states.

For each memory action, Eq. (11) in the paper averages the task outcomes of all its descendant terminal leaves, then forms a local baseline among candidates that share the same parent state. Training uses frozen GPT-5.4 as the reasoner, and gradients update only the memory model.

2. The differences from existing tree methods lie in the scope of operations and the feedback components.

Compared with trajectory-level GRPO, OAT-GRPO moves relative comparisons to the points where the main agent actually calls memory. Compared with Mem-T, which also uses memory-operation trees, we distinguish the following mechanisms:

DimensionMem-T / MoT-GRPOSAM / OAT-GRPO
Memory behavior being optimizedMemory formation, evolution, and retrieval, including retrospective credit assignment to construction operationsBranching to compare recall readouts from the memory module called by the frozen main agent
Propagation of outcome feedbackInternal nodes recursively average the performance of their immediate childrenAverage the outcomes of all descendant leaves of the current action
Local quality signalFeedback such as ground-truth evidence density in retrieved memoryAssess recoverability against committee references, based on the current intent and raw pages

Mem-T has already studied tree-based memory credit assignment. The contribution here should therefore be supported by the effects of SAM's combination of operations and feedback, rather than judged simply by whether it uses a tree.

The role of the recoverability signal is also directly linked to the memory interface. Final failure may result from missing necessary evidence, or from later errors by the main agent after the evidence has been recovered. This auxiliary signal provides a separate assessment of the current readout's quality, while the outcome reward retains the constraint of actual task success.

3. We fix the architecture and sampling budget to test credit-assignment structure and rewards separately.

We fix the SFT initialization and training tasks, and match the rollout generation token budget, including the frozen reasoner. For standard trajectory-level GRPO with mixed rewards, we average the local recoverability scores along the same path and combine this with the final outcome to form a trajectory-level reward. OAT-GRPO instead makes local comparisons at memory calls.

Optimization methodOutcome rewardRecoverability rewardGLM average over four tasks
SFT-only——55.6
Standard trajectory-level GRPO✓—55.9
Standard trajectory-level GRPO✓✓56.2
Tree-local credit assignment✓—56.4
Tree-local credit assignment—✓55.8
Full OAT-GRPO✓✓57.0

With the same mixed reward, tree-local credit assignment improves over trajectory-level GRPO by 0.8 points; with only outcome rewards, it still improves by 0.5 points. With the tree structure fixed, adding recoverability feedback improves the score by 0.6 points, while using recoverability alone lowers it to 55.8. These results separately support the roles of local action comparisons and the two feedback signals, rather than only showing that an extra RL stage helps.

AuthorsOpenReview ↗

[Part 3/4] Response to Reviewer 9KiS

We further keep SAM's branching points, mixed reward, and training budget unchanged and compute outcome feedback by recursively averaging immediate children. This gives 56.8, compared with 57.0 for the full configuration. The two averaging methods are mathematically equivalent on fully expanded trees with a fixed branching factor and equal leaf depth. They may assign different weights to terminal leaves only on unbalanced trees. Thus, this result serves as a comparison with an alternative implementation; the 0.2-point difference alone cannot be attributed to the averaging formula. The roles of local credit assignment and recoverability feedback are tested by the crossed ablations above.

4. The memory mechanism itself also has separate experimental support.

Figure 2 in the paper fixes the write side and page store and compares different read methods. We also add a controlled adaptation of ReadAgent-style access as a baseline.

Read methodQwen BrowseComp
Summary-only39.0%
Recency37.0%
Raw-content38.0%
Online adaptation of ReadAgent-style gist–lookup40.3%
SAM42.2%

In the existing ablation, SAM improves over Raw-content by 4.2 percentage points, showing that reconstruction based on current intent contributes beyond preserving raw pages. Connecting gist–lookup to the same online trajectory interface, with the same Qwen backbone, tools, and total token cap, gives 40.3%, compared with 42.2% for SAM. This controlled adaptation supports the added value of specialized reconstruction on the current task; we do not interpret it as a ranking on the original long-document reading benchmarks. Together with the optimization ablations above, these experiments separately test the memory mechanism and its training method.

Q3: Expert Supervision

You raised concerns about the reliability of teacher supervision and whether the memory model merely imitates GPT-5.4's outputs. The key here is the target space of acceptable recall outputs defined in §2.3: given the same recall intent and raw pages, there is usually more than one valid readout. SFT provides a usable starting behavior, while references from multiple teachers cover a broader range of valid reconstructions, whose value is then tested through downstream task outcomes.

1. The goal is to cover the space of valid recalls, rather than fit a single standard answer.

For the current intent qtq_t and selected pages, different readouts may choose different evidence combinations, structures, and levels of detail while remaining concise, faithful, and informative for later reasoning. The paper denotes this set of acceptable outputs as A⋆(qt,⋅)A^\star(q_t,\cdot).

For example, for the same entity-identification need, one valid readout may highlight time and location constraints, while another may highlight aliases and candidates already ruled out. Both may help the main agent continue solving the task, as long as the information is supported by the raw pages and serves the current need. Thus, using a single text from one teacher as the sole target excludes other valid ways of reconstructing information from supervision.

This space is constrained by the current intent and raw evidence. It describes acceptable recall content, not the teacher model's entire knowledge. We want the memory model to learn to recover useful evidence from pages and adapt to information needs in different states.

Reviewer_aTBpOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

SAM manages long agent trajectories through compact memory cues linked to externally stored raw pages. The main agent selects pages and specifies a recall goal, while a separate memory model extracts relevant information. The memory model is trained with expert supervision and OAT-GRPO, combining downstream task rewards with committee-based recall-quality scores. Experiments report improvements across four benchmarks and two frozen agent backbones.

Critical strengths

Preserving raw trajectories allows agents to revisit details that a summary may omit. The separation between compact cues and detailed records is a practical design for long interactions.

Critical weaknesses
  1. The advantage over ordinary agentic history search is not established. The agent already chooses the pages and recall goal, while the memory model performs query-conditioned summarization. A strong baseline would let the main agent search, read, and summarize its saved interaction history directly. Comparisons against truncation, recent history, and rolling summaries do not establish the benefit over this alternative.

  2. The need for a separately trained memory model is unclear. The paper should compare the trained module with using the existing agent model for the same consolidation and recall operations. The current SFT and RL ablations do not show whether specialization provides enough benefit to justify another model and training pipeline.

  3. The deployment cost is insufficiently evaluated. Appendix B describes separate model endpoints, but does not report end-to-end latency, GPU memory use, serving cost, or utilization. Memory calls are intermittent, so deployment efficiency may depend heavily on request volume and whether the service is shared or loaded on demand. The paper does not establish when this setup is preferable to reusing the existing agent endpoint.

  4. The mixed reward may favor committee preferences over downstream usefulness. The reward assigns coefficients of 0.3 to task outcomes and 0.7 to committee-based recoverability scores. When sibling actions have identical outcome rewards, that component cancels under group centering, leaving the update driven by the proxy score. The paper does not report how often this occurs or whether higher recoverability scores reliably predict better task outcomes. Reward-weight sensitivity and outcome-only versus proxy-only comparisons are needed.

Questions for authors
  1. How does SAM compare with a single agent that searches and summarizes its saved history under the same total inference budget?

  2. What accuracy and cost advantage does the trained memory model provide over using the existing agent model for the same memory operations?

7 comments
AuthorsOpenReview ↗

[Part 1/4] Response to Reviewer aTBp

Thank you for recognizing the value of preserving raw trajectories and the practical benefit of separating compact cues from detailed records. Your concerns focus on four aspects: gains over direct history search, the role of a specialized memory model, the actual cost of a separate service, and the relationship between committee feedback and task utility. We provide the corresponding explanations and experimental results below.

First, we clarify the scope of this paper: SAM handles working memory within a single long-horizon task, which can also be understood as context management. Pages and cues serve the current session and are not passed to other tasks as a cross-session memory store. Your proposed alternatives of active history search and reusing the main model are direct references for testing the practical value of this mechanism.

W1 / Q1: History Access

You pointed out that the main agent can already select pages and specify recall goals, so it should be compared with an approach that lets it directly search, read, and summarize history. Our main explanation is that, once history-search tools are provided, whether the main agent actively uses them and whether the retrieved information effectively supports current decisions are two aspects that need to be tested separately. We first describe the calling behavior observed in practice, then use read ablations and an active-search baseline to explain SAM's design and gains.

1. Providing history-access tools does not mean the main agent will use them actively or sufficiently.

In practice, we found that main agents not specifically trained to use memory tools often do not actively call them, even when given a memory interface. Thus, the direct history-search approach you proposed requires more than the ability to read and summarize. It also depends on the main agent actively deciding when to look back and organizing the corresponding search and processing steps during execution. There remains a practical gap between making tools available and making full use of these capabilities.

This is also one reason we use an external memory module while keeping the main agent frozen: we want to add memory capabilities without retraining the main agent or changing its existing reasoning and tool-use policies. In SAM, the system triggers history consolidation when the threshold is reached and keeps cues in the visible context. When detailed information is needed, the main agent selects pages and states its intent, and the trained memory module handles the actual reconstruction.

Thus, organizing history does not need to wait for the main agent to initiate it, and recall does not require the main agent to carry out the entire history-search and synthesis workflow itself. Recall requests still come from the main agent, so this division of work does not guarantee that all insufficient tool use is resolved. It explains our design motivation; the controlled experiments below address how much it gains over direct history search.

2. The paper's ablations show that, once raw history is accessible, the read method still has a separate effect.

Figure 2 fixes the write side and raw page store and compares different ways of using the same history:

Read methodQwen BrowseComp
Summary-only39.0%
Recency37.0%
Raw-content38.0%
SAM42.2%

SAM improves over Raw-content by 4.2 percentage points and over Summary-only by 3.2 percentage points. In the former comparison, raw information can already be returned; the difference lies in how it is used in the current context. The latter shows that write-time summaries cannot fully cover new needs that arise later. These results therefore support reorganizing information at read time, beyond simply storing more history.

For example, the main agent may have selected pages containing several candidate entities and requested a check of time constraints. Yet the raw records may contain early guesses, candidates later ruled out, and revised evidence together. Page selection determines the information scope, while reconstruction still needs to distinguish the status of these records and return the evidence needed for the current judgment. This is the memory-processing capability that SAM trains.

AuthorsOpenReview ↗

[Part 2/4] Response to Reviewer aTBp

3. The active history-search baseline lets the main agent find and organize evidence itself.

We give the main agent tools for keyword search, semantic search, and reading raw history by page ID, allowing it to search and read multiple times and summarize on its own. The comparison fixes the Qwen main backbone, question set, external search tools, and decoding settings. The total input and output token cap is 1.2M per question for both methods, counting all model calls for task execution and memory processing. Both configurations save and expose the complete raw history produced during their own execution. The active-search baseline's access is not limited by SAM's page selections.

MethodBrowseCompActual total tokens per question
Main agent directly searches, reads, and summarizes history39.8%0.97M
SAM, trained 9B memory model42.2%0.88M

Active history search reaches 39.8%, showing that providing retrieval and summarization tools itself helps the main agent use history. Against this stronger baseline, SAM still improves by 2.4 percentage points while reducing actual token use by about 9.3%. Both methods can access the complete history saved during their own execution and have the same total budget constraint. This difference therefore supports the gains from specialized access and reconstruction, rather than an exclusive ability of SAM to access raw history.

This also addresses your description of the operation as query-conditioned summarization. Summarization based on current needs can describe its form, but whether its output supports later reasoning needs to be tested through actual task performance. This comparison shows that assigning this work to a trained module performs better than leaving all history search and processing to a general-purpose main agent. The next section further fixes the call interface to test the model performing these operations separately.

W2 / Q2: Specialization

You further asked why a separate model needs to be trained, even if reconstruction is useful, rather than reusing the existing main model for the same operations. We keep the cue–page interface and the timing of consolidation and recall unchanged, replace only the model performing memory operations, and compare both accuracy and full-task cost. Unlike the unrestricted history search in the previous section, these configurations use the same memory interface.

The following uses the same Qwen main backbone and BrowseComp question set. The configuration that reuses the main model uses Qwen3.5-35B-A3B for consolidation and recall. The other two use Qwen3.5-9B without SAM training and with SFT + OAT-GRPO training, respectively. All three use the same total token cap, counting inputs, outputs, and model calls across all roles. End-to-end time includes waiting for tools and memory processing.

Model performing memory operationsBrowseCompTotal tokens per questionAverage end-to-end time
Untuned 9B model39.3%0.91M478 seconds
Existing main-agent model40.7%0.95M495 seconds
Trained 9B SAM42.2%0.88M470 seconds

1. The difference between models of the same size shows that specialized training adds real value.

With the same 9B model and call interface, training raises accuracy from 39.3% to 42.2%. This 2.9-percentage-point difference shows that the full effect requires the model to learn how to consolidate and reconstruct history; adding an auxiliary model alone is not enough. Compared with the paper's ablations that remove only SFT or RL, the fully untuned control more directly tests the value of the full training process.

2. Reusing the main model is a valid alternative, but the specialized model has an advantage under the current protocol.

The existing main model achieves 40.7%, higher than the untuned 9B model, showing that a general-purpose model can also perform these operations. The trained 9B model further improves accuracy by 1.5 percentage points, reduces total tokens by about 7.4%, and reduces average end-to-end time by 25 seconds, or about 5.1%. Thus, the separate module's value lies in accuracy and task-execution efficiency under the same interface, rather than just parameter count or the number of modules.

AuthorsOpenReview ↗

[Part 3/4] Response to Reviewer aTBp

3. External training also preserves the main agent's existing capabilities and integration interface.

In practice, we observed that main agents not trained to use memory tools do not always call them actively or sufficiently. SAM therefore lets the system trigger history consolidation once the threshold is reached and places learnable consolidation and reconstruction capabilities in an external module. The main agent selects pages and provides a recall intent when needed.

This allows us to train memory-processing capabilities while keeping the main agent's parameters unchanged. Recall is still initiated by the main agent, and the external module handles the actual information processing. The same trained module can be connected to different main backbones without retraining their reasoning policies separately. This division of work, together with the model-replacement comparison above, explains the purpose of separate training.

W3: Deployment

You noted that the practical value of a separate memory endpoint needs to be assessed through latency, GPU memory, and utilization. Our explanation is that a specialized module can improve execution efficiency per question, but whether an always-on service is cost-effective also depends on having enough requests to share its resources. We provide a set of full deployment statistics below and explain the conditions under which it is useful.

On Qwen BrowseComp, the 9B memory model is kept loaded in BF16 on a single H100 80GB GPU. The following statistics are measured with a single task running. Total tokens and end-to-end time include memory calls; GPU memory includes weights, runtime buffers, and the KV cache.

Total tokens per questionAverage end-to-end timePeak GPU memory of memory endpointAverage memory GPU utilization
0.88M470 seconds26.4 GB6%

1. The separate module's extra calls are already included in the full-task cost. In the same-interface comparison in the previous section, SAM achieves higher accuracy than reusing the main model, with fewer total tokens and lower end-to-end time. Thus, adding a memory module does not necessarily increase the cost of the entire task. The teacher committee and assessor are used only during training and incur no online inference cost.

2. Intermittent calls make a shared service more practical. The average utilization of 6% with a single task indicates substantial idle costs when an always-on endpoint is dedicated to occasional tasks. In this case, reusing an existing main-model endpoint has a resource advantage. Multiple tasks running continuously can instead share the same memory endpoint and spread its fixed resource costs. We therefore position the separate module as a reusable, specialized memory capability and choose the deployment mode based on request load, rather than directly equating fewer tokens per question with lower costs under every load.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.