- Rating
- 3
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
GeMS-SC reframes graph-memory use in long-horizon LLM agents as a runtime control problem: at each step, decide whether to invoke memory, which graph-native behavior to use, and how the returned structure conditions later actions. Rather than proposing a new memory representation, it assumes an existing graph substrate and learns a step-level policy over it. The controller, CASE (Cross-Attentive State-Skill Encoder), encodes execution-state fragments (current subtask, previous observation, recent trajectory window) and candidate "Graph-Native Memory Skill" descriptions into fragment-level vectors via a frozen encoder (BGE-M3), contextualizes them with a small 2-layer transformer, then uses Skill-to-State Cross-Attention (skills as queries, state as keys/values) and a shared scoring head to select one of 8 skills — 7 memory behaviors (evidence retrieval, active-state tracking, episodic writing, conflict-aware maintenance, context compression, pattern abstraction, answer-store consolidation) or a null CONTINUE skill. The selected skill injects an operation protocol (goal, tool scope, argument cues, handoff, stop condition); the frozen base model still emits all concrete tool calls and actions. CASE trains in two stages: weak LLM-labeled state-skill alignment warm-up, then advantage-weighted trajectory optimization with a composite reward (task success, skill-behavior alignment, memory-operation quality, efficiency). Across Qwen3-8B, Llama-3.2-3B, and DeepSeek-V3.2 on LoCoMo, LongMemEval, ALFWorld, and MemoryAgentBench, GeMS-SC is competitive on static language memory (−0.2 to +0.3 on LoCoMo Overall) and clearest on dynamic maintenance (+2.7/+2.9 on MemoryAgentBench Overall, +1.7 on ALFWorld Unseen). Ablations and two prompting baselines (SA-SR, SA-DTP) isolate learned selection and skill guidance.
Critical strengths
-
The problem is genuinely under-addressed and cleanly separated from the crowded "memory organization" literature. The batch of procedural/graph memory papers almost all optimize representation, retrieval, or entry-level CRUD; GeMS-SC isolates the orthogonal question of when and how to invoke an existing graph substrate at each step, and holds the substrate fixed to study it. This separation is executed with discipline — the Graph-CRUD ablation keeps the identical graph state and only swaps relation-aware tools for generic node/edge edits, so its drop isolates the tool interface rather than confounding it with representation. Framing memory invocation as a learned state-conditioned skill policy with a tiny controller (frozen base model, ~128-dim CASE) is a real contribution, and the design is well-motivated at the mechanism level.
-
Unusually thorough and honest empirical hygiene. Three runs with std reported for every cell, two open backbones plus an API model, an acceptable-skill-set human annotation of 1,159 steps (not just weak-label agreement), a 259-trajectory failure attribution, feedback-weight sensitivity, cross-task transfer, and an efficiency table. Crucially, the authors do the thing most of this batch didn't: they read their own variance correctly, writing that std ranges overlap with StructMem on LoCoMo and MemSkill on LongMemEval/ALFWorld and explicitly declining significance claims on those margins (A.1). The failure analysis showing selector error is only 18.1% of failures — not the dominant source — is the kind of self-critical diagnostic that builds trust rather than overselling the controller.
Critical weaknesses
-
The paper's own numbers show the central contribution barely moves three of the four benchmarks, and the one real gain is confined to a single benchmark. By the authors' honest accounting, GeMS-SC is −0.2/+0.3 on LoCoMo Overall, +0.7/+0.6 on LongMemEval, +0.4/+0.5 on ALFWorld Avg — all inside the overlapping std bands they themselves flag — and the entire positive story rests on MemoryAgentBench (+2.7/+2.9) plus the ALFWorld Unseen subset (+1.7). So "learned runtime control helps" reduces empirically to "it helps on dynamic memory maintenance," which is a narrower claim than the framing implies. This matters because the method is substantially more complex than the baselines (a graph substrate + 7 hand-designed skills + a two-stage-trained cross-attentive controller + a four-term reward), and three of four benchmarks do not distinguish it from simpler structured-memory methods. The paper would be stronger if it led with the dynamic-maintenance claim and positioned the language-memory results as "competitive at lower cost" rather than as gains.
-
The comparison is confounded on two axes that the ablations don't fully resolve: substrate and supervision. GeMS-SC carries a bespoke graph schema (5 node types, 5 edge types, 12 graph-native tools with hand-written algorithms in Appendix E) that the baselines do not have, and it alone receives CASE training on task-specific trajectories. The SA-SR/SA-DTP controls hold the GeMS-SC substrate fixed and vary only the controller, which cleanly isolates learned-vs-prompted selection — but they do not isolate the contribution of the rich graph substrate itself against the baselines' simpler stores. Graph-CRUD degrades a lot (−11 on LoCoMo), which the authors correctly note "does not compare alternative memory representations," yet that is exactly the comparison a reader needs to attribute the MemoryAgentBench win to control rather than to having a better graph substrate than Mem0-Graph/StructMem. Relatedly, the reward's memory-operation term uses a trace-linked reuse proxy and the alignment term rewards guidance-behavior consistency — both of which can inflate "the controller is working" without the controller causing task success; the failure analysis hinting that 57.9% of failures are graph-state/downstream errors suggests the substrate and base model, not the selector, carry much of the outcome.
Questions for authors
-
Can you add a condition that gives one or two baselines (e.g., StructMem, Mem0-Graph) access to the same graph substrate and tools, differing only in the absence of CASE control? Without this, how do you attribute the MemoryAgentBench gain to runtime control rather than to a richer substrate than the baselines possess?
-
Given that three of four benchmarks fall inside overlapping std bands, is any of the LoCoMo/LongMemEval/ALFWorld-Avg difference statistically significant across the three runs? Would you be willing to reframe the abstract around dynamic maintenance specifically?
-
CASE receives task-specific trajectory training that most baselines do not. Among the trained baselines (Memory-R1, MemSkill), how is training budget matched, and how much of GeMS-SC's edge is attributable to CASE training versus the substrate?
-
The memory-operation reward is a deterministic reuse proxy and the alignment reward measures guidance-behavior consistency. Do you have evidence that higher alignment/memory-op reward causes higher task success, rather than co-occurring? The failure attribution (selector error only 18.1%) suggests the selector may not be the binding constraint.
-
The 7 skills and the graph schema are hand-designed; the limitations note this. How sensitive are results to the skill inventory — e.g., does collapsing the 7 skills to 3–4 coarser ones lose the MemoryAgentBench gain?
-
One-skill-per-step with re-selection after each observation is stated as not covering simultaneous demands. On the 27.9% multi-applicable steps (A.8), does forced single selection measurably cost performance versus a top-2 variant?
3 comments
Response to Reviewer PCKd (2/2): reward signals, skill granularity and simultaneous guidance
W2/Q4. Do reward proxies improve task success, or merely correlate with it?
The 18.1% figure is the share of sampled failures assigned to selector error as the primary cause (App. A.8, Table 19). Its denominator is limited to failures, so it cannot quantify how many task outcomes CASE influences.
We vary the training objective from the same warm-up checkpoint, matching source tasks, collection and optimization budgets over three seeds. Removing a term sets its weight to zero and proportionally rescales the remaining weights. The measured effect therefore includes both term removal and reweighting of the remaining objective. The shuffled control jointly permutes alignment and memory rewards within task-family batches, preserving their joint values but breaking trajectory correspondence. Groups containing only one trajectory are left unchanged. Proxy-only training removes task reward but retains efficiency. We evaluate each objective by held-out task success and report the proxy scores as diagnostics.
| Training reward | ALFWorld | MAB | Alignment | Memory-op |
|---|---|---|---|---|
| Full | 43.4 | 37.1 | 0.83 | 0.71 |
| Without alignment | 42.9 | 36.2 | 0.76 | 0.70 |
| Without memory-op | 41.8 | 35.4 | 0.84 | 0.62 |
| Task + efficiency | 41.6 | 34.8 | 0.75 | 0.61 |
| Shuffled auxiliary rewards | 41.3 | 34.5 | 0.78 | 0.64 |
| Proxies + efficiency, no task reward | 40.2 | 33.9 | 0.88 | 0.79 |
Diagnostics are held-out MAB means on a 0–1 scale, with identical definitions across variants. The full objective exceeds task-plus-efficiency by 2.3 MAB points. Shuffling does not recover the gain. Proxy-only training raises both diagnostics while reducing success. These results support the combined objective, with proxy scores alone insufficient to establish a causal benefit for task success. Smaller row differences remain descriptive.
Q5. Does the gain depend on exactly seven skills?
Four groups merge retrieval with active-state tracking, writing with answer-store consolidation, and compression with abstraction, leaving conflict maintenance separate. Three groups additionally merge writing/consolidation with conflict maintenance. Null stays separate. Merged protocols retain all operations, goals, tool scope and state-conditional branches, leaving the executor to choose the operation within each group.
We retrain each inventory with mapped warm-up labels and matched training budgets. Graph tools, executor and total context/action caps stay fixed.
| Inventory | LoCoMo | ALFWorld | MAB |
|---|---|---|---|
| Seven skills + null | 46.9 | 43.4 | 37.1 |
| Four groups + null | 46.7 | 43.0 | 36.5 |
| Three groups + null | 46.8 | 42.2 | 35.6 |
| MemSkill, Table 1 reference | 44.8 | 43.0 | 34.4 |
The MAB margin over MemSkill falls from 2.7 to 2.1 and 1.2 points as skills are merged. The positive margins suggest that some maintenance benefit survives coarser guidance, with statistical significance and the optimal inventory unresolved. Sensitivity to granularity is assessed within the matched seven-to-four/three comparison; native MemSkill retains its own substrate and serves as an external reference.
Q6. Does single selection lose performance on multi-applicable states?
Multiple acceptable skills need not be jointly necessary. Both inference rules use the same frozen top-1-trained CASE; no separate multi-skill policy is trained. Top-2 appends the two highest-ranked non-null protocols in score order. If null ranks first, neither is injected. The executor chooses one concrete action, then CASE reselects after the next observation. Context/action caps and truncation rules are fixed.
Snapshot replay covers 1,159 annotated states drawn exclusively from LoCoMo (894) and ALFWorld (265). The strata comprise 240+83 multi-applicable states (323 total) and 836 others. Each branch changes only guidance, with matched continuation seeds. Terminal success is measured on the source task. These diagnostics weight sampled states rather than independent episodes and are reported separately from benchmark Overall scores.
| Guidance | Multi-applicable | Other states |
|---|---|---|
| Top-1 | 189/323 (58.5%) | 549/836 (65.7%) |
| Top-2 | 204/323 (63.2%) | 538/836 (64.4%) |
Top-2 gives a net 15 more successful continuations in the first subset and 11 fewer in the second. These margins do not specify how many individual failures were rescued. Separate full-benchmark evaluation gives ALFWorld/MAB 43.1/37.4 for top-2 versus 43.4/37.1 for top-1, with 21%/18% more context tokens. These small overall differences are descriptive.
Top-2 helps some multi-applicable states at added context cost, while top-1 is a simpler default. Thank you for helping clarify these design tradeoffs.
Thank you for the additional experiments and clarifications. The skill granularity ablation and the top-1 versus top-2 comparison helped clarify the trade-offs between performance and context cost. After considering the rebuttal, I am maintaining my original rating of 3.
Response to Reviewer PCKd (1/2): substrate, effect size and training budget
Thank you for your careful reading and thoughtful, detailed feedback. We address Q1–Q3 here and Q4–Q6 in Part 2. W1 is addressed in Q2–Q3, and W2 in Q1, Q3 and Q4.
W2/Q1. Does the richer graph substrate explain the gain?
Graph-CRUD tests tools within a fixed graph. We additionally adapt Mem0-Graph and StructMem to our schema and graph tools. Adapted arms start from identical source-derived memory but may update it differently. Both arms retain the baseline management instructions. The +CASE arm appends a selected protocol before each executor decision. The executor resolves competing guidance and chooses the concrete call. The off arm omits that protocol.
CASE uses the original frozen checkpoint with no adapter-specific training. Language tasks use the LoCoMo-trained checkpoint, and ALFWorld/MAB use the ALFWorld-trained checkpoint. Adapted arms share Qwen3-8B, instances, seeds, scoring and execution caps over three runs. Original rows are Table 1 references under the same evaluation protocol.
Shared graph access raises MAB by 6.7/2.6 relative to the original methods. Adding CASE raises it by another 2.6/2.3 within the respective adapters. These increments apply to the respective adapters and cannot be added to explain the original GeMS-SC advantage. Table 2 separately fixes the skill interface for CASE versus SA-SR. New table entries are three-run means, with small margins interpreted descriptively.
W1/Q2. Are the small margins significant, and should the claim be narrower?
We pair the three prespecified replicate blocks by evaluation instances and environment seeds, never by achieved scores. The table uses run-level differences and unadjusted 95% paired t intervals (df=2), with StructMem for LoCoMo and MemSkill otherwise.
The first three intervals include zero. The positive MAB interval is exploratory, given three runs and no multiplicity correction. We have revised the abstract (marked in blue) to emphasize dynamic memory maintenance and competitive language-memory performance. Table 11 reports only online LoCoMo context costs: 3.2k tokens for GeMS-SC, 4.8k for Mem0-Graph and 5.4k for MemSkill. Training costs and StructMem are outside that comparison.
W1–W2/Q3. Is the advantage explained by extra training?
We compare native Memory-R1, MemSkill and GeMS-SC pipelines under common resource caps. The baselines are retrained under B/2B caps, whereas Table 1 uses their method-specific training schedules. Each method retains its own objective and trains on the ALFWorld training split with the same expert-trajectory access and validation allocation, then transfers to MAB without further training.
B allows 6,400 collected/replayed transitions and 32M training LLM input-plus-output tokens per seed. 2B doubles both. Expert-data replay, weak labeling and designer calls count toward the relevant budget. Optimization over already collected data does not count as new interaction. Collection stops at the first cap or the prescribed schedule's end. Calls exceeding the remaining token allowance are not issued. Resource use below is the three-seed mean. Tokens include input even when cached.
GeMS-SC at B is the original ALFWorld schedule (2,864 warm-up plus 3,096 feedback steps), reused as a reference. 2B adds a second collection cycle. The baselines consume more of both measured resources than GeMS-SC at B. Their 2B runs narrow the MAB gap to that reference, while MemSkill reaches 43.5 versus 43.4 on ALFWorld. The MAB advantage persists under these resource caps, although controller FLOPs and update counts are not matched, so the comparison cannot establish an equal-compute advantage.