acceptodds
Under review as a conference paper at ICLR 2027

GeMS-SC: Learning a Cross-Attentive State-Skill Policy for Runtime Graph-Memory Control in Long-Horizon Agents

Abstract

Long-horizon LLM agents with graph-structured memory must decide at every execution step whether memory is needed, which graph-native behavior to invoke, and how returned structures should condition later actions. Existing approaches primarily optimize memory representation, retrieval, or generic entry-level operations, leaving this runtime control problem under-specified. We formulate graph-memory invocation as a state-conditioned skill-selection policy and introduce GeMS-SC. Its controller, the Cross-Attentive State-Skill Encoder (CASE), jointly contextualizes execution-state fragments, including the current subtask, previous observation, and recent trajectory, with candidate Graph-Native Memory Skills. A lightweight interaction encoder followed by Skill-to-State Cross-Attention constructs state-conditioned skill representations; a shared scoring head then selects one skill or Continue Regular Execution. The selected skill injects an operation protocol specifying the memory objective, graph-tool scope, argument cues, result handoff, and stopping condition, while the frozen base agent retains control over concrete tool calls and task actions. CASE is learned in two stages: weak state-skill alignment from future memory-tool behavior, followed by advantage-weighted trajectory optimization using task success, skill-behavior alignment, memory-operation quality, and execution efficiency. Controlled comparisons with state-aware LLM skill routing and direct graph-tool prompting isolate the effects of learned selection and intermediate skill guidance. Across Qwen3-8B-Instruct and Llama-3.2-3B-Instruct on LoCoMo, LongMemEval, ALFWorld, and MemoryAgentBench, GeMS-SC is competitive with the strongest structured-memory baseline on LoCoMo and yields consistent gains on cross-session and unseen interaction metrics, with its largest improvement on dynamic memory maintenance.

Primary area:
foundation or frontier models, including LLMs
Keywords:
LLM agents, graph-structured long-term memory, state-conditioned skill selection, runtime memory invocation, long-horizon decision-making

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_PCKdOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

GeMS-SC reframes graph-memory use in long-horizon LLM agents as a runtime control problem: at each step, decide whether to invoke memory, which graph-native behavior to use, and how the returned structure conditions later actions. Rather than proposing a new memory representation, it assumes an existing graph substrate and learns a step-level policy over it. The controller, CASE (Cross-Attentive State-Skill Encoder), encodes execution-state fragments (current subtask, previous observation, recent trajectory window) and candidate "Graph-Native Memory Skill" descriptions into fragment-level vectors via a frozen encoder (BGE-M3), contextualizes them with a small 2-layer transformer, then uses Skill-to-State Cross-Attention (skills as queries, state as keys/values) and a shared scoring head to select one of 8 skills — 7 memory behaviors (evidence retrieval, active-state tracking, episodic writing, conflict-aware maintenance, context compression, pattern abstraction, answer-store consolidation) or a null CONTINUE skill. The selected skill injects an operation protocol (goal, tool scope, argument cues, handoff, stop condition); the frozen base model still emits all concrete tool calls and actions. CASE trains in two stages: weak LLM-labeled state-skill alignment warm-up, then advantage-weighted trajectory optimization with a composite reward (task success, skill-behavior alignment, memory-operation quality, efficiency). Across Qwen3-8B, Llama-3.2-3B, and DeepSeek-V3.2 on LoCoMo, LongMemEval, ALFWorld, and MemoryAgentBench, GeMS-SC is competitive on static language memory (−0.2 to +0.3 on LoCoMo Overall) and clearest on dynamic maintenance (+2.7/+2.9 on MemoryAgentBench Overall, +1.7 on ALFWorld Unseen). Ablations and two prompting baselines (SA-SR, SA-DTP) isolate learned selection and skill guidance.

Critical strengths
  1. The problem is genuinely under-addressed and cleanly separated from the crowded "memory organization" literature. The batch of procedural/graph memory papers almost all optimize representation, retrieval, or entry-level CRUD; GeMS-SC isolates the orthogonal question of when and how to invoke an existing graph substrate at each step, and holds the substrate fixed to study it. This separation is executed with discipline — the Graph-CRUD ablation keeps the identical graph state and only swaps relation-aware tools for generic node/edge edits, so its drop isolates the tool interface rather than confounding it with representation. Framing memory invocation as a learned state-conditioned skill policy with a tiny controller (frozen base model, ~128-dim CASE) is a real contribution, and the design is well-motivated at the mechanism level.

  2. Unusually thorough and honest empirical hygiene. Three runs with std reported for every cell, two open backbones plus an API model, an acceptable-skill-set human annotation of 1,159 steps (not just weak-label agreement), a 259-trajectory failure attribution, feedback-weight sensitivity, cross-task transfer, and an efficiency table. Crucially, the authors do the thing most of this batch didn't: they read their own variance correctly, writing that std ranges overlap with StructMem on LoCoMo and MemSkill on LongMemEval/ALFWorld and explicitly declining significance claims on those margins (A.1). The failure analysis showing selector error is only 18.1% of failures — not the dominant source — is the kind of self-critical diagnostic that builds trust rather than overselling the controller.

Critical weaknesses
  1. The paper's own numbers show the central contribution barely moves three of the four benchmarks, and the one real gain is confined to a single benchmark. By the authors' honest accounting, GeMS-SC is −0.2/+0.3 on LoCoMo Overall, +0.7/+0.6 on LongMemEval, +0.4/+0.5 on ALFWorld Avg — all inside the overlapping std bands they themselves flag — and the entire positive story rests on MemoryAgentBench (+2.7/+2.9) plus the ALFWorld Unseen subset (+1.7). So "learned runtime control helps" reduces empirically to "it helps on dynamic memory maintenance," which is a narrower claim than the framing implies. This matters because the method is substantially more complex than the baselines (a graph substrate + 7 hand-designed skills + a two-stage-trained cross-attentive controller + a four-term reward), and three of four benchmarks do not distinguish it from simpler structured-memory methods. The paper would be stronger if it led with the dynamic-maintenance claim and positioned the language-memory results as "competitive at lower cost" rather than as gains.

  2. The comparison is confounded on two axes that the ablations don't fully resolve: substrate and supervision. GeMS-SC carries a bespoke graph schema (5 node types, 5 edge types, 12 graph-native tools with hand-written algorithms in Appendix E) that the baselines do not have, and it alone receives CASE training on task-specific trajectories. The SA-SR/SA-DTP controls hold the GeMS-SC substrate fixed and vary only the controller, which cleanly isolates learned-vs-prompted selection — but they do not isolate the contribution of the rich graph substrate itself against the baselines' simpler stores. Graph-CRUD degrades a lot (−11 on LoCoMo), which the authors correctly note "does not compare alternative memory representations," yet that is exactly the comparison a reader needs to attribute the MemoryAgentBench win to control rather than to having a better graph substrate than Mem0-Graph/StructMem. Relatedly, the reward's memory-operation term uses a trace-linked reuse proxy and the alignment term rewards guidance-behavior consistency — both of which can inflate "the controller is working" without the controller causing task success; the failure analysis hinting that 57.9% of failures are graph-state/downstream errors suggests the substrate and base model, not the selector, carry much of the outcome.

Questions for authors
  1. Can you add a condition that gives one or two baselines (e.g., StructMem, Mem0-Graph) access to the same graph substrate and tools, differing only in the absence of CASE control? Without this, how do you attribute the MemoryAgentBench gain to runtime control rather than to a richer substrate than the baselines possess?

  2. Given that three of four benchmarks fall inside overlapping std bands, is any of the LoCoMo/LongMemEval/ALFWorld-Avg difference statistically significant across the three runs? Would you be willing to reframe the abstract around dynamic maintenance specifically?

  3. CASE receives task-specific trajectory training that most baselines do not. Among the trained baselines (Memory-R1, MemSkill), how is training budget matched, and how much of GeMS-SC's edge is attributable to CASE training versus the substrate?

  4. The memory-operation reward is a deterministic reuse proxy and the alignment reward measures guidance-behavior consistency. Do you have evidence that higher alignment/memory-op reward causes higher task success, rather than co-occurring? The failure attribution (selector error only 18.1%) suggests the selector may not be the binding constraint.

  5. The 7 skills and the graph schema are hand-designed; the limitations note this. How sensitive are results to the skill inventory — e.g., does collapsing the 7 skills to 3–4 coarser ones lose the MemoryAgentBench gain?

  6. One-skill-per-step with re-selection after each observation is stated as not covering simultaneous demands. On the 27.9% multi-applicable steps (A.8), does forced single selection measurably cost performance versus a top-2 variant?

3 comments
AuthorsOpenReview ↗

Response to Reviewer PCKd (1/2): substrate, effect size and training budget

Thank you for your careful reading and thoughtful, detailed feedback. We address Q1–Q3 here and Q4–Q6 in Part 2. W1 is addressed in Q2–Q3, and W2 in Q1, Q3 and Q4.

W2/Q1. Does the richer graph substrate explain the gain?

Graph-CRUD tests tools within a fixed graph. We additionally adapt Mem0-Graph and StructMem to our schema and graph tools. Adapted arms start from identical source-derived memory but may update it differently. Both arms retain the baseline management instructions. The +CASE arm appends a selected protocol before each executor decision. The executor resolves competing guidance and chooses the concrete call. The off arm omits that protocol.

CASE uses the original frozen checkpoint with no adapter-specific training. Language tasks use the LoCoMo-trained checkpoint, and ALFWorld/MAB use the ALFWorld-trained checkpoint. Adapted arms share Qwen3-8B, instances, seeds, scoring and execution caps over three runs. Original rows are Table 1 references under the same evaluation protocol.

PipelineLoCoMoLongMemEvalALFWorldMAB
Mem0-Graph, original42.844.237.126.9
Mem0-Graph, shared graph/tools44.747.140.633.6
Same adapted pipeline + CASE45.048.042.736.2
StructMem, original47.147.340.531.9
StructMem, shared graph/tools47.048.742.134.5
Same adapted pipeline + CASE47.249.643.236.8
GeMS-SC46.952.843.437.1

Shared graph access raises MAB by 6.7/2.6 relative to the original methods. Adding CASE raises it by another 2.6/2.3 within the respective adapters. These increments apply to the respective adapters and cannot be added to explain the original GeMS-SC advantage. Table 2 separately fixes the skill interface for CASE versus SA-SR. New table entries are three-run means, with small margins interpreted descriptively.

W1/Q2. Are the small margins significant, and should the claim be narrower?

We pair the three prespecified replicate blocks by evaluation instances and environment seeds, never by achieved scores. The table uses run-level differences and unadjusted 95% paired t intervals (df=2), with StructMem for LoCoMo and MemSkill otherwise.

Qwen3-8B benchmarkGeMS-SC minus comparator95% interval
LoCoMo-0.2[-2.0, 1.6]
LongMemEval+0.7[-1.4, 2.8]
ALFWorld Avg.+0.4[-1.3, 2.1]
MAB Overall+2.7[0.6, 4.8]

The first three intervals include zero. The positive MAB interval is exploratory, given three runs and no multiplicity correction. We have revised the abstract (marked in blue) to emphasize dynamic memory maintenance and competitive language-memory performance. Table 11 reports only online LoCoMo context costs: 3.2k tokens for GeMS-SC, 4.8k for Mem0-Graph and 5.4k for MemSkill. Training costs and StructMem are outside that comparison.

W1–W2/Q3. Is the advantage explained by extra training?

We compare native Memory-R1, MemSkill and GeMS-SC pipelines under common resource caps. The baselines are retrained under B/2B caps, whereas Table 1 uses their method-specific training schedules. Each method retains its own objective and trains on the ALFWorld training split with the same expert-trajectory access and validation allocation, then transfers to MAB without further training.

B allows 6,400 collected/replayed transitions and 32M training LLM input-plus-output tokens per seed. 2B doubles both. Expert-data replay, weak labeling and designer calls count toward the relevant budget. Optimization over already collected data does not count as new interaction. Collection stops at the first cap or the prescribed schedule's end. Calls exceeding the remaining token allowance are not issued. Resource use below is the three-seed mean. Tokens include input even when cached.

PipelineCapTransitionsTokens (M)ALFWorldMAB
Memory-R1B6,40027.640.434.1
Memory-R12B12,80056.941.134.6
MemSkillB6,21431.943.335.2
MemSkill2B12,48763.843.535.6
GeMS-SCB5,96021.443.437.1
GeMS-SC2B11,92043.743.837.4

GeMS-SC at B is the original ALFWorld schedule (2,864 warm-up plus 3,096 feedback steps), reused as a reference. 2B adds a second collection cycle. The baselines consume more of both measured resources than GeMS-SC at B. Their 2B runs narrow the MAB gap to that reference, while MemSkill reaches 43.5 versus 43.4 on ALFWorld. The MAB advantage persists under these resource caps, although controller FLOPs and update counts are not matched, so the comparison cannot establish an equal-compute advantage.

AuthorsOpenReview ↗

Response to Reviewer PCKd (2/2): reward signals, skill granularity and simultaneous guidance

W2/Q4. Do reward proxies improve task success, or merely correlate with it?

The 18.1% figure is the share of sampled failures assigned to selector error as the primary cause (App. A.8, Table 19). Its denominator is limited to failures, so it cannot quantify how many task outcomes CASE influences.

We vary the training objective from the same warm-up checkpoint, matching source tasks, collection and optimization budgets over three seeds. Removing a term sets its weight to zero and proportionally rescales the remaining weights. The measured effect therefore includes both term removal and reweighting of the remaining objective. The shuffled control jointly permutes alignment and memory rewards within task-family batches, preserving their joint values but breaking trajectory correspondence. Groups containing only one trajectory are left unchanged. Proxy-only training removes task reward but retains efficiency. We evaluate each objective by held-out task success and report the proxy scores as diagnostics.

Training rewardALFWorldMABAlignmentMemory-op
Full43.437.10.830.71
Without alignment42.936.20.760.70
Without memory-op41.835.40.840.62
Task + efficiency41.634.80.750.61
Shuffled auxiliary rewards41.334.50.780.64
Proxies + efficiency, no task reward40.233.90.880.79

Diagnostics are held-out MAB means on a 0–1 scale, with identical definitions across variants. The full objective exceeds task-plus-efficiency by 2.3 MAB points. Shuffling does not recover the gain. Proxy-only training raises both diagnostics while reducing success. These results support the combined objective, with proxy scores alone insufficient to establish a causal benefit for task success. Smaller row differences remain descriptive.

Q5. Does the gain depend on exactly seven skills?

Four groups merge retrieval with active-state tracking, writing with answer-store consolidation, and compression with abstraction, leaving conflict maintenance separate. Three groups additionally merge writing/consolidation with conflict maintenance. Null stays separate. Merged protocols retain all operations, goals, tool scope and state-conditional branches, leaving the executor to choose the operation within each group.

We retrain each inventory with mapped warm-up labels and matched training budgets. Graph tools, executor and total context/action caps stay fixed.

InventoryLoCoMoALFWorldMAB
Seven skills + null46.943.437.1
Four groups + null46.743.036.5
Three groups + null46.842.235.6
MemSkill, Table 1 reference44.843.034.4

The MAB margin over MemSkill falls from 2.7 to 2.1 and 1.2 points as skills are merged. The positive margins suggest that some maintenance benefit survives coarser guidance, with statistical significance and the optimal inventory unresolved. Sensitivity to granularity is assessed within the matched seven-to-four/three comparison; native MemSkill retains its own substrate and serves as an external reference.

Q6. Does single selection lose performance on multi-applicable states?

Multiple acceptable skills need not be jointly necessary. Both inference rules use the same frozen top-1-trained CASE; no separate multi-skill policy is trained. Top-2 appends the two highest-ranked non-null protocols in score order. If null ranks first, neither is injected. The executor chooses one concrete action, then CASE reselects after the next observation. Context/action caps and truncation rules are fixed.

Snapshot replay covers 1,159 annotated states drawn exclusively from LoCoMo (894) and ALFWorld (265). The strata comprise 240+83 multi-applicable states (323 total) and 836 others. Each branch changes only guidance, with matched continuation seeds. Terminal success is measured on the source task. These diagnostics weight sampled states rather than independent episodes and are reported separately from benchmark Overall scores.

GuidanceMulti-applicableOther states
Top-1189/323 (58.5%)549/836 (65.7%)
Top-2204/323 (63.2%)538/836 (64.4%)

Top-2 gives a net 15 more successful continuations in the first subset and 11 fewer in the second. These margins do not specify how many individual failures were rescued. Separate full-benchmark evaluation gives ALFWorld/MAB 43.1/37.4 for top-2 versus 43.4/37.1 for top-1, with 21%/18% more context tokens. These small overall differences are descriptive.

Top-2 helps some multi-applicable states at added context cost, while top-1 is a simpler default. Thank you for helping clarify these design tradeoffs.

Thank you for the additional experiments and clarifications. The skill granularity ablation and the top-1 versus top-2 comparison helped clarify the trade-offs between performance and context cost. After considering the rebuttal, I am maintaining my original rating of 3.

Reviewer_rJnWOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper focuses on how to enable long-horizon AI agents to use and maintain memory more rationally at every step. The authors propose GeMS-SC, which adds a trainable 'memory scheduler' on top of existing graph-structured memory systems. This scheduler selects appropriate memory operation guidance based on the current task and recent events, which is then executed by the large language model. This paper assumes the system is already equipped with a graph-structured memory that organizes entities, facts, states, and evidence into nodes connected by relations such as 'support,' 'update,' and 'conflict.' This allows the system to not only retrieve a specific piece of information but also track its origin, its relationships with other information, and its current validity. This paper conducts experiments on several benchmarks with Qwen-8B-instruct, and the results reveal the advantage in dynamic memory maintenance.

Critical strengths

The most notable contribution is designing the usage pattern of long-term memory as a separately trainable, step-by-step decision-making control module that plugs into the exterior of the base large language model.

It provides a clear system division of labor:

  • Graph memory is responsible for maintaining states, relationships, and evidence;
  • Predefined skills describe different types of memory requirements;
  • CASE determines which type of guidance is most appropriate for the current context;
  • The base large language model executes specific tool calls and task actions.

From an engineering perspective, this offers a promising direction for improving existing graph-memory agents: the memory scheduling module can be optimized independently without needing to retrain the base large language model in this approach.

Critical weaknesses

However, the paper also outlines several clear limitations:

  • The skill set is manually designed. Currently, there are only seven memory skills, with one selected per step. Addressing new domain requirements may require expanding this skill set.
  • The framework relies on existing graph-structured memory. The authors do not demonstrate that graph memory is superior to text memory, event logs, or other storage formats.
  • Selecting the correct guidance does not guarantee successful execution. Errors in tool parameters, graph relationships, or subsequent reasoning can still compromise the final outcome.
  • The authors did not systematically evaluate system performance under corrupted or contaminated graph memory, and coverage of long-term real-world deployment remains limited.
Questions for authors
  1. Can authors report the results by applying GeMS-SC with GPT or Claude?
  2. It seems that graph memory is not superior to text memory.
  3. Although the authors claim train-test separation, critical experimental details are insufficient for independent verification. Specifically, they need to provide the LoCoMo data version, complete conversation IDs, QA IDs, and split code, along with the ALFWorld training task list.
5 comments
AuthorsOpenReview ↗

Response to Reviewer rJnW (1/2): verifiable splits and graph versus text memory

Thank you for your questions about the evaluation and the source of the gains. We also appreciate your recognition of the independently trainable controller with a frozen base model. This part answers Q3 and W2/Q2. Part 2 covers executors, skills, execution failures and robustness.

Q3. Data identities, split code and training lists.

The supplementary index docs/reviewer_rjnw_q3.md links the five requested items. LoCoMo is partitioned at conversation level: conv-26 for warm-up, conv-30/41 for feedback, and conv-42/43/44/47/48/49/50 for evaluation. The evaluation manifest contains 1,155 category-1–4 QA, with category counts 641/208/231/75. Its conversations do not overlap the training source conversations. Training-query counts are reported in App. F.1, Table 41.

Requested informationSupplementary material
LoCoMo versionPinned source revision, SHA-256 and download/verification code
Conversation IDsComplete conversation list and stage assignment
QA IDsStage-specific source QA lists with original array positions, eligibility flags and content hashes
Split codeDeterministic split and overlap-check scripts
ALFWorld training tasksAll 320 training task identities, alongside the 242/85 Seen/Unseen lists

The LoCoMo QA identifiers combine the original conversation identity and QA array position, so each entry can be traced to the source file. ALFWorld lists retain task/trial identities. The 320/242/85 counts refer to the packaged data version. The supplement also provides aggregation tools keyed by sample ID, with explicit benchmark-specific weighting rules.

W2/Q2. Does the gain come from graph storage or from control?

Our focus is on controlling memory operations. As stated in the Introduction (p. 2, ll. 072–074) and Limitations, the study does not aim to establish a general ranking of storage formats. Graph memory lets us operate directly on relations. Bounded traversal recovers multi-hop evidence, update edges track the active state, and conflict edges preserve the provenance of superseded facts (Apps. C, E). AriGraph and Zep reflect the same direction toward relational, evolving agent memory. Graph-CRUD keeps the same graph and replaces these tools with generic node/edge edits. It loses 11.6 points on LoCoMo and 11.9 on MAB (Table 2), supporting the value of the operation interface.

To examine learned control on a second substrate, we built a structured-text memory from the same source information, with indexed records (content, entities, timestamps, source, revision) and retrieval/write/update/delete tools. Skill protocols are mapped to this interface and CASE is retrained on it. Executor, instances and budgets are matched (Qwen3-8B):

Memory / controllerLoCoMoLongMemEvalALFWorldMAB
Text / SA-SR42.948.637.231.4
Text / CASE45.852.140.634.3
Graph / SA-SR43.849.239.033.2
Graph / CASE46.952.843.437.1

Learned control improves performance on both substrates: CASE exceeds SA-SR by 2.9–3.5 points on text and 3.1–4.4 on graph. With CASE, the graph advantage is small on static QA (1.1 and 0.7). It reaches 2.8 on both ALFWorld and MAB, where state updates and conflict handling are needed. Details are in new Appendix A.14.

AuthorsOpenReview ↗

Response to Reviewer rJnW (2/2): GPT, Gemini and Claude executors, skills, execution failures and robustness

The submission already evaluates Qwen3-8B, Llama-3.2-3B and DeepSeek-V3.2 (Apps. A.2, A.5). Under DeepSeek-V3.2, GeMS-SC ties StructMem on LoCoMo and leads on LongMemEval (69.1 vs. 67.7) and MAB (50.2 vs. 47.4). Following your suggestion, we added GPT-4o, Gemini 2.5 Flash and Claude 3.7 Sonnet. GeMS-SC uses the Qwen-trained CASE without retraining. Each baseline keeps its own memory pipeline with trained components frozen. Within each executor, instances, judge, decoding and execution limits are matched. Gemini thinking and Claude extended thinking are disabled. We ran each setting three times:

ExecutorMethodLoCoMoLongMemEvalALFWorldMAB
GPT-4oA-MEM66.955.247.835.6
Mem0-Graph70.358.750.438.9
StructMem75.861.650.942.7
MemSkill72.664.354.644.5
GeMS-SC75.566.457.148.3
Gemini 2.5 FlashA-MEM68.457.351.237.4
Mem0-Graph72.161.555.340.8
StructMem77.663.954.744.2
MemSkill74.366.259.146.9
GeMS-SC77.967.660.449.8
Claude 3.7 SonnetA-MEM70.258.657.438.7
Mem0-Graph74.562.360.142.5
StructMem79.465.458.845.9
MemSkill76.268.163.548.2
GeMS-SC79.070.966.852.4

Across all three executors, GeMS-SC stays within 0.4 points of StructMem on LoCoMo and leads on LongMemEval, ALFWorld and MAB. Its MAB margins are 3.8/2.9/4.2 points (App. A.13, Table 24). With the graph and tools fixed, frozen CASE exceeds SA-SR by 4.1–5.3 on MAB. Executor-specific retraining adds 0.3/0.4/0.2 points for GPT/Gemini/Claude, each below 0.5 (Table 25).

W1. Manual skills and one selection per step. The hand-designed inventory may need extension for new domains. CASE reselects after each observation. 38.7–46.2% of trajectories use multiple skills sequentially (App. A.8).

The extension setup withholds Conflict-Aware Maintenance, its slot and guidance from both training stages, then adds it with 200 adaptation steps:

AdaptationALFWorldMAB
Skill withheld41.234.1
Insert, no updates41.634.9
New slot only42.135.6
Slot + scoring head42.836.5
All trainable CASE parameters43.236.9
Skill present from start (Table 2)43.437.1

In this experiment, slot-and-head adaptation recovers 80% of the MAB gain. The added protocol uses fixed graph tools and a fixed executor; new tool semantics and automatic skill discovery remain outside this experiment (App. A.11).

W3. Correct guidance does not guarantee correct execution. CASE chooses the guidance, and the executor issues the calls; errors can arise at either stage (Sec. 3.1). App. A.8 (Table 19) attributes 259 failed trajectories to the selector (47), tool execution (62), graph state (73) and downstream reasoning or action (77). We replayed each of the 47 selector failures from the same saved state. One branch kept the original choice. The other used an annotated acceptable skill. Both branches succeeded in 4 cases. Only the corrected branch succeeded in 24, only the unchanged branch in 1, and neither in 18. This gives 28 versus 5 successes. Correcting selection improves recovery, while the remaining failures show that execution and downstream reasoning still matter.

W4. Corrupted memory and long-term use. We injected each corruption type into 20% of its eligible MAB memory objects (fixed seed), giving both controllers identical corrupted memories:

MAB OverallSA-SRCASE
Clean (Table 2)33.237.1
Stale active states28.734.6
Conflicting relations29.433.9
Deleted evidence26.127.3

CASE keeps its advantage under every type and loses least under stale states (2.5 points), consistent with active-state maintenance. Deleted evidence hurts both most because the information is gone (new Appendix A.12). We also evaluated 100 memory streams, with 20 events per session and 20 delayed queries per stream at each horizon:

Delayed-query accuracy10 sessions50100
SA-SR77.464.849.6
CASE79.271.558.3

The gap grows from 1.8 to 8.7 points across 10 to 100 sessions, supporting stronger retention in this controlled long-stream setting. These controlled streams do not establish real-world deployment performance.

Thank you again for your feedback. If the additional results and clarifications address your concerns, we would appreciate your reconsideration of the rating. We welcome any remaining questions.

Thank you for the additional experiments and clarifications. After considering the rebuttal, I would raise my rating to 3.

Reviewer_k2HBOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces GeMS-SC, a framework that trains a dedicated controller to dynamically invoke graph-structured memory skills at runtime. At each execution step, the controller evaluates the current subtask, previous observation, and execution history to select one of seven hand-crafted memory skills or a null option that provides no guidance. Each skill supplies a structured protocol defining its objective, tool preferences, argument construction, handoff format, and termination criteria, while a frozen base agent executes the actual tool calls, environment actions, and final responses. The controller, named CASE, combines frozen text embeddings, state-skill cross-attention, and a shared scoring head to model state-skill interactions. It is trained via weak-label state-skill alignment followed by trajectory-level feedback optimization that penalizes inefficient operations and rewards task completion, memory-op quality, and guidance adherence.

Critical strengths
  1. The same-substrate prompting controls meaningfully isolate the value of learned runtime selection.

  2. The evaluation is broad and includes useful transfer and diagnostic analyses.

Critical weaknesses
  1. Incomplete Positioning Against Runtime Graph-Memory Controllers

The distinction between GeMS-SC and prior runtime graph-memory methods that also focus on how graph-structured memory is invoked needs further discussion. AriGraph integrates graph maintenance directly into an observation-action loop, HAGE [1] learns sequential graph traversal, ExpGraph [2] trains a graph-retrieval controller for a frozen executor, [3] learns graph-based selection of execution-guiding strategies from downstream utility. Further experiments might be needed as well to demonstrate the effectiveness of the proposed methods.

  1. Fixed Interface Limits Practical Generalization

While cross-task transfer are investigated in Appendix, they rely on a completely static interface with the same base model, graph tools, and candidate skills. Because CASE uses index embeddings tied to eight pre-defined skills, the experiments demonstrate routing across new task distributions rather than true adaptation to new memory operations. It remains unclear whether adding an unseen skill or modifying existing tool semantics requires full controller retraining. Testing the fixed controller with a different base executor or an extended skill inventory would help establish whether the learned policy reflects general, description-based matching or merely fixed-routing logic.

[1] Jiang, D., Li, Y., Li, G., Li, Q., & Li, B. (2026). Hage: Harnessing agentic memory via rl-driven weighted graph evolution. arXiv preprint arXiv:2605.09942.

[2] Feng, T., Ye, C., Luo, T., Xu, J., Xu, X., Zhang, H., ... & You, J. (2026). ExpGraph: Model-Agnostic Experience Learning with Graph-Structured Memory for LLM Agents. arXiv preprint arXiv:2605.30712.

[3] Xia, S., Xu, Z., Chai, J., Fan, W., Song, Y., Wang, X., ... & Wang, J. (2025). From experience to strategy: Empowering llm agents with trainable graph memory. arXiv preprint arXiv:2511.07800.

Questions for authors
  1. What is the precise methodological advance over learned memory-skill selection and prior use of graph memory to guide execution?

  2. How well does the trained CASE controller generalize to different backbone models or memory skill inventories?

  3. One of the paper's main claims is that GeMS-SC dynamically selects memory skills based on the current execution state, rather than relying only on the task description. However, it is unclear how much this step-by-step adaptation contributes to the performance gains. Could the authors compare the full CASE controller with a version that selects skills using only the current task or subtask, without access to recent observations or interaction history, while keeping other settings the same? This would help clarify whether the improvements mainly come from learning which memory skills are useful for a task, or from dynamically adapting skill selection as the task progresses.

3 comments
AuthorsOpenReview ↗

Response to Reviewer k2HB (1/2): contribution of execution state

Thank you for recognizing learned selection and asking how execution state contributes. This part answers Q3 on execution-state inputs. Part 2 addresses the methodological advance and generalization to other executors and skill inventories.

Q3. Does step-wise state matter beyond knowing the task?

Step-wise state matters most when the appropriate memory operation changes during an episode. TaskOnly receives only the task or question, while SubtaskOnly receives only the current subtask. Neither sees observations or history. Both are retrained with the same data, two-stage procedure, skill inventory, frozen Qwen3-8B executor and budget as full CASE. We ran each setting three times:

Selector inputLoCoMoLongMemEvalALFWorldMAB
SA-SR, full state, prompted (Table 3)43.849.239.033.2
TaskOnly, retrained46.251.140.334.2
SubtaskOnly, retrained46.651.941.735.3
Full CASE (Table 3)46.952.843.437.1

Learning improves performance even without execution state: the trained TaskOnly selector outperforms the prompted SA-SR, which sees the full state, on all four benchmarks.

State inputs contribute most on tasks requiring memory maintenance. Full CASE exceeds TaskOnly by 0.7 and 1.7 points on LoCoMo and LongMemEval. The gaps reach 3.1 on ALFWorld and 2.9 on MAB, where object states change during an episode. SubtaskOnly recovers part of the gap because subtasks partly reflect progress.

Appendix A.10 details the matched state-input comparison. Appendix A.8 (Table 18) separately counts trajectories that use multiple non-null skills sequentially; it does not measure how often skills switch between steps.

AuthorsOpenReview ↗

Response to Reviewer k2HB (2/2): methodological advance and interface generalization

W1/Q1. What is the methodological advance over prior runtime graph-memory controllers?

Prior work already learns graph-memory control. Our contribution is step-wise operation-protocol selection with joint state–skill modeling. At each action–observation step, CASE uses the current subtask, observation and history (Secs. 3.1–3.3). Sec. 2.2 compares the control units:

MethodWhat is learned or controlled
AriGraphLLM-driven graph update and retrieval inside the planning loop, without a learned selector
HAGELearned query-conditioned traversal and edge representations
ExpGraphLearned retrieval controls and graph updates for a frozen executor
Xia et al.Graph weights over strategies, used to train the agent itself
MemSkillSkill selection for memory construction, with an evolving skill bank
GeMS-SCStep-wise graph-operation guidance via joint state–skill modeling

MemSkill also learns state-conditioned skill selection and evolves its library. CASE uses a fixed inventory, jointly contextualizes state and skill fragments, and applies Skill-to-State Cross-Attention to select retrieval or maintenance guidance.

Further experiments. We retrained two CASE ablations with the same data and budget as full CASE. RetrievalOnly may choose only the retrieval skill or null. The pooled scorer separately encodes the same state and descriptions, without joint contextualization or cross-attention.

Qwen3-8B controllerLoCoMoLongMemEvalALFWorldMAB
RetrievalOnly46.552.140.934.0
Pooled scorer46.151.741.535.4
Full CASE (Table 3)46.952.843.437.1

RetrievalOnly loses 3.1 MAB points, mainly in maintenance: LRU 36.1 → 31.0, SF 25.6 → 20.1, versus AR 49.8 → 49.1. The pooled scorer scores below CASE on every benchmark (up to 1.9 on ALFWorld), consistent with the original MAB ablations: 32.4 without CASE learning and 31.6 without cross-attention (App. A.6). Against MemSkill, GeMS-SC leads MAB by 2.7 and LongMemEval by 0.7 (Table 1), with the same ordering under Llama and DeepSeek-V3.2. The internal controls are detailed in new Appendix A.10.

W2/Q2. Generalization across executors and skill inventories.

Description and slot dependence.

CASE scores candidates from both skill descriptions and index-slot embeddings (App. B.2, Table 28). With CASE fixed, we tested three interventions: paraphrased descriptions (three predefined sets), permuted non-null slot embeddings (five derangements of seven slots), and identical neutral descriptions:

Selector inputALFWorldMAB
Original (Table 2)43.437.1
Paraphrased descriptions43.336.9
Permuted slot embeddings40.733.8
Neutral descriptions39.632.9

Scores remain stable under meaning-preserving paraphrases, changing by at most 0.2. Removing description content or scrambling slots costs 3.3–4.2 on MAB, indicating that both contribute to selection.

Adding a new skill. To test extension, we withheld Conflict-Aware Maintenance, including its slot and guidance, from both training stages and added it afterward with 200 update steps:

Adding a withheld skillALFWorldMAB
Skill withheld41.234.1
Inserted with its description, no updates41.634.9
Adapt the new slot only42.135.6
Adapt slot and scoring head42.836.5
Adapt all trainable CASE parameters43.236.9
Trained with the skill from the start (Table 2)43.437.1

For this held-out protocol, slot-and-head adaptation recovers 80% of the MAB gap to full training. All-parameter adaptation recovers 93%. Both extend the skill inventory over unchanged tools, so the results concern protocol adaptation within the existing interface (App. A.11).

Executor and task transfer.

  • Executors. The frozen Qwen-trained CASE transfers to GPT-4o, Gemini 2.5 Flash and Claude 3.7 Sonnet. It exceeds the strongest independent baseline on MAB by 2.9–4.2 points across the three executors. On the same graph and tools, retraining adds 0.3/0.4/0.2 MAB points for GPT/Gemini/Claude (App. A.13, Table 25).
  • Task distributions. A LoCoMo-trained CASE applied directly to MAB scores 36.1, still above MemSkill's 34.4 (App. A.7, Table 16).

Thank you for your questions about the contribution and generalization. We welcome further discussion of these comparisons.

AuthorsOpenReview ↗

Follow-up on our response

Dear Reviewer k2HB,

Thank you again for your review. Before the discussion period closes, we would like to briefly recap our two-part response above.

  • Execution state. Selectors without observations or history trail full CASE by up to 3.1 points, mainly on tasks where states change. See App. A.10.
  • Advance over prior controllers. Step-wise protocol selection with joint state–skill modeling, supported by retrieval-only and pooled-scorer ablations. See Sec. 2.2 and App. A.10.
  • Generalization. A new skill recovers 80% of the gap with light adaptation, and the controller transfers to GPT, Gemini and Claude without retraining. See App. A.11 and A.13.

We hope this summary makes our earlier responses easier to check, and we would be grateful to know whether they address your questions.

Comments on the paper

2 comments
AuthorsOpenReview ↗

Summary of responses, additional results, and paper revisions

We thank the AC and reviewers for their detailed feedback. k2HB recognized the same-substrate controls, and rJnW highlighted the independently trainable controller with a frozen base model. This note summarizes the additional results and revisions. We address each question in the individual responses.

GeMS-SC learns to select graph-memory guidance for a frozen executor at each step. The controlled comparisons show additional gains from learned, state-dependent selection, with the largest improvements on tasks requiring memory maintenance. The controller also transfers to GPT, Gemini and Claude executors without retraining.

Results and clarifications

  1. Data identity materials (rJnW). Versioned data sources and checksums, conversation/QA manifests, partition checks, ALFWorld lists and aggregation code are included. The index docs/reviewer_rjnw_q3.md links each requested item to its file and verification command. See rJnW Part 1.

  2. State inputs (k2HB). Retrained TaskOnly trails full CASE by 3.1 points on ALFWorld, 2.9 on MAB and 0.7 on LoCoMo. For SubtaskOnly, the corresponding gaps are 1.7, 1.8 and 0.3 points. Both selectors still exceed the prompted router. k2HB Part 1, App. A.10.

  3. GPT, Gemini and Claude executors (rJnW, k2HB). We evaluated GPT-4o, Gemini 2.5 Flash and Claude 3.7 Sonnet. The frozen Qwen-trained CASE exceeds the strongest independent baseline on MAB by 2.9–4.2 points across the three executors. On LoCoMo, it stays within 0.4 of StructMem. rJnW Part 2, App. A.13.

  4. Positioning (k2HB). The comparison with prior learned graph-memory controllers identifies step-wise operation-protocol selection and joint state–skill modeling as our contribution. Internal controls show a 3.1-point MAB loss with retrieval-only guidance and lower scores on every benchmark with a separate pooled scorer. k2HB Part 2, Sec. 2 and App. A.10.

  5. Graph versus text (rJnW). CASE improves over the prompted router on both substrates. The graph's advantage is largest on maintenance tasks. rJnW Part 1, App. A.14.

  6. Skills, failures and robustness (rJnW, k2HB). Description and slot interventions show that both inputs contribute. Adapting a held-out skill recovers 80% of the MAB gap to full training. Paired replay increases successes from 5 to 28 among 47 selector failures. In the long-stream evaluation, the delayed-query advantage grows from 1.8 to 8.7 points across 10 to 100 sessions. See k2HB Part 2 and rJnW Part 2, with skill and corruption diagnostics in Apps. A.11–A.12.

  7. Sources of gain and scope of the claim (PCKd). With graph and tool access shared, adding CASE raises MAB by 2.6/2.3 points within the adapted Mem0-Graph/StructMem pipelines. Doubling the baselines’ training caps narrows their MAB gaps to GeMS-SC at the original budget, but the gaps remain positive. The small LoCoMo, LongMemEval and ALFWorld Avg. differences do not establish statistical significance. We have narrowed the abstract to emphasize dynamic memory maintenance and competitive language-memory performance. See PCKd Part 1; reward and skill-design controls are in Part 2.

Paper revisions are marked in blue.

AuthorsOpenReview ↗

Summary of the discussion for the AC

Dear Area Chair,

Thank you for handling our submission. Our earlier general response summarizes the new results and revisions. Here we summarize the reviewers' assessments and how the discussion concluded.

Shared strengths. k2HB noted that the same-substrate controls "meaningfully isolate the value of learned runtime selection". PCKd found the problem "genuinely under-addressed" and the empirical work "unusually thorough and honest". rJnW noted that the framework "provides a clear system division of labor" among graph memory, predefined skills, the CASE selector and the base model.

Reviewer rJnW, final rating 3, asked for results with GPT or Claude executors, a comparison of graph and text memory, and verifiable data splits. The controller transfers to GPT-4o, Gemini 2.5 Flash and Claude 3.7 Sonnet without retraining and exceeds the strongest baseline on MAB by 2.9–4.2 points. We also compared graph and text substrates in App. A.14 and provided split manifests and code in the supplement. After considering the rebuttal, the reviewer acknowledged the additional experiments and clarifications.

Reviewer PCKd, final rating 3, raised possible confounds from the graph substrate and training budget. With shared graph and tools, adding CASE within Mem0-Graph/StructMem raises MAB by 2.6/2.3 points, and doubling the baselines' training caps narrows but does not close the gap. The abstract now highlights dynamic memory maintenance, where the gains are largest. The reviewer acknowledged that the new ablations clarified the design trade-offs.

Reviewer k2HB, final rating 3, asked about the role of execution state, positioning, and generalization to other executors and skills. The executor question overlaps with rJnW's and is answered by the same transfer results. Selectors without observations or history trail full CASE by up to 3.1 points, and a new skill recovers 80% of the gap with light adaptation. See App. A.10, A.11 and A.13. The reviewer has not yet posted a follow-up.

GeMS-SC shows that learned, step-wise selection of graph-memory guidance improves frozen executors and transfers across model families without retraining. Thank you again for your time.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.