acceptodds
Under review as a conference paper at ICLR 2027

Transition-Parametric Memory for Language Agents: Reversible Compiled Memory Fields

Abstract

Long-horizon language agents can reuse past experience, but combining record-level editability with compact memory access remains challenging. Textual memories typically require retrieval and in-context processing, while experience absorbed into shared parameters can be difficult to audit or selectively remove. We formulate **transition-parametric memory**, which preserves each decision transition as an identifiable record with a separate parametric contribution. We introduce **Reversible Compiled Memory Fields** (RCMF) to aggregate these contributions into a fixed-shape field queried by the current agent state to modulate a frozen language model. Online field reads avoid textual replay and have computational cost independent of memory count. Individual records can be inserted or removed through additive updates without retraining the model or recompiling other records. Experiments on AppWorld, ALFWorld, and WebShop show favorable task performance against multiple baselines. Core implementation code is provided in Appendix D.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Language Agents, Agent Memory, Parametric Memory

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_NuojOpenReview ↗
Rating
2
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

The paper studies parametric memory for language agents: how to keep every experience record individually traceable and editable while reading the memory as a whole at constant cost. The proposed RCMF framework stores each decision transition as a provenance-tracked record that contributes an independent additive term to a fixed-shape compiled memory field; reads are produced by a frozen query module and injected into a frozen Qwen3-8B, and records are inserted or removed by adding or subtracting their contributions without retraining. On AppWorld, ALFWorld, and WebShop, with baselines rerun in the same environments, RCMF attains the highest average success (30.96% vs. 27.33% for the strongest baseline); compiled reads stay below 0.16 ms while explicit per-record reads grow to 30.29 ms at 499 records.

Critical strengths
  1. The unification of record-level editability with constant-cost parametric access holds by construction: each record's contribution is independently attributable and can be added or subtracted exactly, the compiled read equals the explicit per-record sum term by term (Eqs. 12-14), and the query-key decomposition introduces no additional projection (Appendix A.3). Reversible editing, provenance auditing, and fixed-shape reads thus become algebraic properties of the framework -- a substantive contribution relative to Larimar, MemGen, and δ\delta-mem -- and the main support for the overall assessment.
  2. The evidence is layered and the comparison fair: four baselines are rerun under the same backbone, environments, and evaluators; ablations isolate state-conditioned addressing as the largest empirical factor (-12.50pp); and field-level shuffles with record-level log-probability interventions (related +0.1901 vs. unrelated +0.0025) support the mechanistic explanation beyond end-to-end scores.
Critical weaknesses
  1. The headline results lack stability evidence. Table 1 reports single runs on small fixed evaluation sets (168/134/200 tasks) with no repetitions or interval estimates; the margin over the strongest baseline corresponds to only 2-7 tasks per benchmark (AppWorld 48 vs. 41; ALFWorld 52 vs. MemGen's 50), and the 3.63pp average margin is of the same order as the ~3.3pp binomial standard error of a single proportion at n=168, so the stability of the advantage cannot be judged -- this directly limits the effectiveness claim.
  2. The most distinctive selling points have the weakest evidence. Record-level editing reversibility is supported only algebraically and through one-step log-probability shifts on 20+20 held-out pairs (Eq. 18), with no closed-loop experiment on task success after insertion/removal; "N-independence" covers only read latency and active storage (Table 9), and superposition interference or addressing-accuracy degradation of the fixed-shape field as N grows is never measured behaviorally (the deployment bank holds only 499 records).
Questions for authors
  1. Are all Table 1 results single runs? Could you provide multi-seed repetitions, or at least paired interval estimates over the evaluation sets (e.g., paired bootstrap), to judge whether the 3.63pp average margin over the strongest baseline exceeds run-to-run variability?
  2. Have you run (or could you add) an end-to-end editing evaluation: after deleting or replacing several compiled records, does success on the affected tasks change as algebraic reversibility predicts? This is the minimal evidence for the practical utility of record-level editing.
  3. How many transitions do the deployment memory banks for ALFWorld and WebShop contain, and how many supervised states were used for Writer-Reader training on each? Appendix B.3 fully specifies only the AppWorld instance (366 states, 499 records).
  4. As N grows well beyond 499, is the addressing quality of the compiled field (related-record log-probability shift or downstream success) preserved, or are only latency and storage constant?
3 comments
AuthorsOpenReview ↗

Response to Reviewer Nuoj (Part 1)

W1: Thank you for the constructive feedback. The final RCMF design was completed only one month before submission, leaving insufficient time for repeated runs. We continued these experiments after submission and have now obtained three runs for all five methods (Table 1-1). All applicable training stages were rerun from scratch. RCMF achieves the highest mean on every reported metric. Its lowest observed success rate also exceeds the strongest baseline’s highest observed rate on each benchmark, although the ALFWorld margin remains small.

Table 1-1. Effectiveness across three runs (mean ± SD).

MethodWebShop RewardWebShop Success (%)AppWorld Success (%)ALFWorld Success (%)Average Success (%)
ReAct0.5815 ± 0.002219.83 ± 0.5819.44 ± 0.6931.84 ± 0.4323.71 ± 0.28
ReMe0.5825 ± 0.003422.17 ± 0.7623.41 ± 0.9132.59 ± 0.8626.06 ± 0.81
MemGen0.5734 ± 0.006018.17 ± 1.2619.64 ± 0.6035.82 ± 1.4924.54 ± 0.79
δ-mem0.6004 ± 0.003722.17 ± 1.0424.40 ± 1.1935.07 ± 0.7527.22 ± 0.98
RCMF0.6339 ± 0.001725.67 ± 0.2928.77 ± 0.3439.05 ± 1.1431.16 ± 0.58

Table 1-1 measures run-to-run variability, whereas Table 1-2 assesses uncertainty in the performance differences arising from task sampling. We compute paired 95% bootstrap confidence intervals by jointly resampling the same evaluation tasks for RCMF and the strongest original baseline.

Table 1-2. Paired uncertainty against the strongest original baselines.

MetricRCMF (reported)Strongest original baselineDifferencePaired 95% CI
AppWorld success48/168δ-mem: 41/168+4.17 pp[+0.595, +8.333] pp
ALFWorld success52/134MemGen: 50/134+1.49 pp[0.000, +3.731] pp
WebShop success51/200δ-mem: 45/200+3.00 pp[+0.500, +6.000] pp
WebShop mean reward0.6320δ-mem: 0.6017+0.0303[+0.0132, +0.0499]

The paired intervals exclude zero for AppWorld success and both WebShop metrics, supporting positive task-level differences in the reported runs. The ALFWorld CI reaches zero, leaving its two-task gain statistically uncertain. Overall, repeated runs support consistent improvements, with weaker statistical evidence on ALFWorld.

W2: Thank you for the insightful feedback. We address W2 in two parts: W2a Behavioral Memory Editing and W2b Memory Growth and Interference.

W2a: We conducted behavioral editing on 20 related and 20 unrelated AppWorld tasks using records from the 98 transitions withheld, extending the original log-probability probes to end-to-end task outcomes. Related tasks are grouped by correctness of target action under Background, regardless of final task success. Each task uses the same model, decision state, and source record under 4 conditions: Background (401 records), Insert with Original (adding the original record), Replace with Misleading (replacing the original record with semantically misleading version), and Delete to Background (removing inserted record). Original and misleading records never coexist.

We report A (percentage of tasks for which correct action is executed at matched decision state) and T (full-task success from independent rollouts). Observed rates are in %; brackets give task-paired 95% bootstrap CIs for differences from Background (pp).

Table 2-1. Effects of memory editing on action accuracy and task success.

GroupBackgroundInsert OriginalReplace MisleadingDelete to Background
Related-Incorrect (10)A: 0 [—]; T: 20 [—]A: 80 [+50,+100]; T: 70 [+10,+90]A: 0 [0,0]; T: 10 [−30,0]A: 0 [0,0]; T: 20 [0,0]
Related-Correct (10)A: 100 [—]; T: 90 [—]A: 100 [0,0]; T: 90 [0,0]A: 20 [−100,−50]; T: 30 [−90,−30]A: 100 [0,0]; T: 90 [0,0]
Unrelated (20)A: 50 [—]; T: 60 [—]A: 50 [0,0]; T: 65 [0,+15]A: 50 [0,0]; T: 60 [−15,+15]A: 50 [0,0]; T: 60 [0,0]

Inserting original record corrects 8/10 wrong related actions while preserving correct ones. Replacing it with misleading content disrupts related decisions, whereas unrelated tasks remain largely unaffected. Deletion restores Background performance. These results provide behavioral evidence of memory specificity beyond log-probability probes.

AuthorsOpenReview ↗

Response to Reviewer Nuoj (Part 2)

W2b: Using the same frozen model and evaluation settings, we tested memory growth across 4 nested banks: 401-record Writer–Reader optimization pool, 499 records (adding 98 withheld transitions without weight updates), and 800 / 1600 records (adding 301 and 1101 LLM-generated synthetic variants to the 499-record bank to evaluate performance with larger banks). These variants modify entities or descriptions in training-side transitions while preserving their action semantics, structure, and provenance.

Table 2-2. Task success and retention under memory-bank growth.

Memory CountAppWorld SuccessRetained Successes from N=401
40144/16844/44
49948/16842/44
80050/16841/44
160050/16843/44

Retention counts how many of the 44 tasks solved at N=401 remain successful as the bank grows, exposing regressions despite higher overall success. All learned components remain frozen. Performance improves beyond the original Writer–Reader training-bank size, while most previously solved tasks are retained. These results provide behavioral evidence that useful memory access survives substantial synthetic interference.

Q1: Yes. All results in the original Table 1 were based on a single run. Details can be found in W1 and Tables 1-1 and 1-2.

Q2: Addressed in W2a and Table 2-1.

Q3: As described in Appendix B.3, AppWorld uses 366 supervised decision states and a memory pool of 401 transitions from 29 training-split tasks. 8 additional tasks from the same training split provide 98 held-out transitions. After Writer–Reader optimization, the 98 held-out transitions are added to the memory field without further training, yielding 499 deployment records.

Table 5-1 provides the corresponding statistics for ALFWorld and WebShop. The same Writer–Reader training framework and optimization settings are used across benchmarks, while data-dependent quantities, such as the number of training units and optimizer updates, vary accordingly.

Table 5-1. Memory construction and Writer–Reader training statistics.

ParameterAppWorldALFWorldWebShop
Tasks for training2959101
Transitions from training tasks401408411
Eligible supervised decision states366390384
Held-out tasks81423
Transitions from held-out tasks9899105
Total deployment transitions499507516

Q4: Addressed in W2b and Table 2-2.

Area_Chair_MnayOpenReview ↗

Please respond to the author rebuttal

Dear Reviewer Nuoj, could you please respond to the author's rebuttal? Thanks!

Reviewer_VoVTOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes transition-parametric memory: each agent decision transition (state–action–observation plus provenance metadata) is treated as an identifiable atomic memory record, and each record contributes an independently additive component to parametric memory. To operationalize this formalism, the authors propose Reversible Compiled Memory Fields (RCMF):

  • The Writer maps the four semantic sections of a record (goal/state/action/observation) × two readouts (mean/final pooling)—eight representations in total—into eight payload slots Ci∈R8×dpC_i \in \mathbb{R}^{8\times d_p}.
  • The Selector (an ensemble with J=3J=3) produces state–transition scores, and this bilinear score can be exactly decomposed as score(s,mi)=q(s)⊤ki+bscore(s,m_i)=q(s)^\top k_i + b (Eq. 10, App. A.3), so the addressing coefficient is ϕ(s,mi)=q(s)⊤ki\phi(s,m_i)=q(s)^\top k_i.
  • The Field Compiler accumulates the outer products ρiki⊗Ci\rho_i k_i \otimes C_i over all records into a fixed-shape field AM∈R960×8×256A_M \in \mathbb{R}^{960\times 8\times 256}. By distributivity, the compiled read (Eq. 14) is mathematically equivalent to explicit per-record reading (Eq. 12) under real arithmetic, and the serving shape is independent of NN.
  • The readout is passed through RMSNorm and then injected by a Cross-Attention Reader into layers 7/14/21/28 of frozen Qwen3-8B; it does not enter the prompt or KV cache.

Records can be inserted or deleted by adding or subtracting ΔAi\Delta A_i without retraining. Experiments on AppWorld/ALFWorld/WebShop compare RCMF with ReAct, ReMe, MemGen, and δ\delta-mem; RCMF achieves the highest average success rate (30.96%) and includes ablations (Table 3), mechanistic interventions (Table 4: key–payload / state–query shuffling and record-level log-prob analysis), and efficiency and NN-scaling measurements (Tables 2/9/10).

Critical strengths
  • S1. The problem is clearly positioned and practically valuable. The paper accurately characterizes the tension between textual memory (human-readable and directly editable, but requiring retrieval and in-context processing and competing with the current task for context) and parametric memory (compact access, but multiple experiences are mixed through shared parameter updates, making them hard to audit and selectively remove). Proposing an additive compiled field that simultaneously preserves record identity and compact access is a reasonable and novel design space.
  • S2. The core technique is elegant and correct, and the derivations are complete and credible. The exact query–key decomposition of the selector ensemble (App. A.3, Eqs. 21–27) is the key technical trick: it allows the addressing coefficient to be precomputed offline on the key side and computed online on the query side, and makes the compiled read strictly equivalent to the explicit sum (the element-wise derivation in App. A.4). This is why O(1) serving (relative to NN) and exact reversible editing can hold simultaneously.
  • S3. The leakage-control design is rigorous. The task-legal field (Eq. 17) excludes records from the current task during training and checkpoint selection; the evaluation population is frozen and is not used for memory construction or any method configuration (App. B.1); and the archival memory ledger is separated from the compiled serving tensor (App. A.6).
Critical weaknesses
  • W1. A key control is missing: the same addressing mechanism with top-kk raw text injection. The paper’s motivation includes the advantage of parametric access over textual replay, but the main experimental table lacks a baseline that uses the same Selector for addressing and injects the selected transitions as raw text into the prompt.
  • W2. The “reversible” claim lacks behavioral evidence. The 98 post-training records have not been subjected to an end-to-end editing experiment (insert → measure success rate → delete → measure recovery); existing evidence consists only of algebraic equivalence and log-prob probes. Moreover, although records in the Writer–Reader training pool can be algebraically subtracted, their influence on the training parameters during joint Writer–Reader optimization cannot be removed by field subtraction, which limits the scope of “reversible.”
  • W3. The three benchmarks effectively correspond to three independently trained systems; no Writer/Reader transfer experiment is conducted, and the Selector is also pretrained using in-benchmark data.
  • W4. The effect of memory-bank growth is not validated. NN-scaling is measured only for latency, not success rate. The Reader and slot-wise RMSNorm are trained only under the field distribution induced by 401 records. The risk of crowding when NN is much larger than 401 is not examined; thus, “continually growing memory” currently remains a mechanism-level claim.
  • W5. The backbone is only Qwen3-8B, and key hyperparameters are given without selection rationale or ablation, e.g., Reader layers 7/14/21/28, J=3J=3, and dp=256d_p=256.
Questions for authors
  • Q1. Can the authors add a control that uses the same frozen Selector to select top-kk transitions and inject them as raw text (i.e., the πraw\pi_{raw} in App. B.3 as a deployable policy), and report success rates on all three benchmarks? This would isolate the net benefit of parametric access relative to textual replay.
  • Q2. Can the authors conduct an end-to-end editing experiment using some post-training records: insert → measure success rate → delete → measure recovery?
  • Q3. The main results are single-seed; because the benchmarks are small, the gap between RCMF and some baselines such as δ\delta-mem is only 5–7 questions. Can the authors add multi-seed variance?
2 comments
AuthorsOpenReview ↗

Response to Reviewer VoVT (Part 1)

Thank you for the constructive feedback. Our responses are provided below.

W1: We evaluated the raw-text policy on all 168 AppWorld tasks using the paper’s evaluation configuration. At each step of its own rollout, it uses the same frozen Selector and eligible bank as RCMF, selects one transition by the teacher’s class-then-exemplar rule, and inserts its text into the prompt. It has no access to expert test actions or beneficial/harmful labels. We use (k=1) because this directly evaluates the single-transition (πraw)(\pi_{\rm raw}) policy defined in Appendix B.3; RCMF itself reads the full eligible field and has no retrieval (k) to match. We additionally measured complete online time to first token (TTFT), including state encoding, query construction, memory access, and model prefill. Short, Medium, and Long are the same three AppWorld prompts as Table 10 in our paper (7326, 9999, and 16985 tokens, respectively). We are extending this comparison to ALFWorld and WebShop.

Table 1-1. AppWorld task success and speed across memory policies.

PolicySuccess (%)Short (ms)Medium (ms)Long (ms)
ReAct (no memory)19.05330464880
Raw-text replay27.38123615862734
RCMF (compiled)28.576679381775

Raw replay captures much of the improvement over ReAct, confirming the value of selected experience. 7 tasks were solved only by RCMF and 5 only by raw replay. The paired difference was +1.19 pp (95% CI [−2.98, +5.36]), so we do not establish a task-success advantage for compilation over replay. RCMF did, however, reduce median online TTFT by 35–46% on the three tested states.

W2: We conducted behavioral editing on 20 related and 20 unrelated AppWorld tasks using records from the 98 transitions withheld, extending the original log-probability probes to end-to-end task outcomes. Related tasks are grouped by correctness of target action under Background, regardless of final task success. Each task uses the same model, decision state, and source record under 4 conditions: Background (401 records), Insert with Original (adding the original record), Replace with Misleading (replacing the original record with semantically misleading version), and Delete to Background (removing inserted record). Original and misleading records never coexist.

We report A (percentage of tasks for which correct action is executed at matched decision state) and T (full-task success from independent rollouts). Observed rates are in %; brackets give task-paired 95% bootstrap CIs for differences from Background (pp).

Table 2-1. Effects of memory editing on action accuracy and task success.

GroupBackgroundInsert OriginalReplace MisleadingDelete to Background
Related-Incorrect (10)A: 0 [—]; T: 20 [—]A: 80 [+50,+100]; T: 70 [+10,+90]A: 0 [0,0]; T: 10 [−30,0]A: 0 [0,0]; T: 20 [0,0]
Related-Correct (10)A: 100 [—]; T: 90 [—]A: 100 [0,0]; T: 90 [0,0]A: 20 [−100,−50]; T: 30 [−90,−30]A: 100 [0,0]; T: 90 [0,0]
Unrelated (20)A: 50 [—]; T: 60 [—]A: 50 [0,0]; T: 65 [0,+15]A: 50 [0,0]; T: 60 [−15,+15]A: 50 [0,0]; T: 60 [0,0]

Inserting original record corrects 8/10 wrong related actions while preserving correct ones. Replacing it with misleading content disrupts related decisions, whereas unrelated tasks remain largely unaffected. Deletion restores Background performance. These results provide behavioral evidence of memory specificity beyond log-probability probes.

We agree that subtracting a record's compiled contribution cannot remove its historical influence on Writer–Reader parameters during optimization. As clarified in Appendix A.6, reversibility refers to deployment-time editing of explicit memory contributions after Writer–Reader optimization, not training-time unlearning. Our experiment uses records withheld from optimization, demonstrating behavioral insertion and deletion without retraining the model.

W3: Thank you for highlighting this important distinction. Each benchmark uses its own Selector, pretrained on that benchmark’s training split, and a separately trained Writer–Reader pair. The three evaluations therefore test the same architecture and compilation mechanism across environments, not the transfer of learned parameters between them. To clarify this limitation, we will explicitly acknowledge the lack of cross-benchmark parameter transfer in the Conclusion and highlight it as an important direction for future work.

AuthorsOpenReview ↗

Response to Reviewer VoVT (Part 2)

W4: Using the same frozen model and evaluation settings, we tested memory growth across 4 nested banks: 401-record Writer–Reader optimization pool, 499 records (adding 98 withheld transitions without weight updates), and 800 / 1600 records (adding 301 and 1101 LLM-generated synthetic variants to the 499-record bank to evaluate performance with larger banks). These variants modify entities or descriptions in training-side transitions while preserving their action semantics, structure, and provenance.

Table 4-1. Task success and retention under memory-bank growth.

Memory CountAppWorld SuccessRetained Successes from N=401
40144/16844/44
49948/16842/44
80050/16841/44
160050/16843/44

Retention counts how many of the 44 tasks solved at N=401 remain successful as the bank grows, exposing regressions despite higher overall success. All learned components remain frozen. Performance improves beyond the original Writer–Reader training-bank size, while most previously solved tasks are retained. These results provide behavioral evidence that useful memory access survives substantial synthetic interference.

W5: We agree that evaluating a second backbone and explaining the architectural hyperparameters would strengthen the empirical evidence. To address this concern, we conducted a sensitivity analysis of Reader layer placement on AppWorld, while additional experiments on backbone generalization, Selector ensemble size, and payload dimensionality are currently underway.

We chose layers 7, 14, 21, and 28 to space the four Readers evenly and center them around the midpoint of the backbone. We tested two alternatives by shifting all four Readers four layers earlier or later while preserving their spacing. For each alternative, we kept the Selector fixed and jointly retrained the Writer and Reader (Table 5-1).

Table 5-1. Reader layer sensitivity on AppWorld.

Reader LayersAppWorld Success
7/14/21/28 (default)48/168
3/10/17/2446/168
11/18/25/3244/168

The default configuration achieves slightly higher success than both alternative placements, suggesting a potential benefit of centering the Reader layers across the backbone depth. For the remaining hyperparameters, (J=3) corresponds to three independently trained Selector checkpoints, while (d_p=256) is chosen to balance payload capacity against the compiled-field footprint. We are currently conducting sensitivity analyses for (J) and (d_p), along with an evaluation on an additional 8B-scale backbone, and will incorporate the corresponding results into the revised manuscript.

Q1: Addressed in W1 and Table 1-1 (AppWorld). We are extending this comparison to ALFWorld and WebShop.

Q2: Addressed in W2 and Table 2-1.

Q3: The final RCMF design was completed in the month before submission, leaving insufficient time for repeated runs. We continued these experiments after submission and have now obtained three runs for all five methods (Table 8-1). All applicable training stages were rerun from scratch. RCMF achieves the highest mean on every reported metric. Its lowest observed success rate also exceeds the strongest baseline’s highest observed rate on each benchmark, although the ALFWorld margin remains small.

Table 8-1. Effectiveness across three runs (mean ± SD).

MethodWebShop RewardWebShop Success (%)AppWorld Success (%)ALFWorld Success (%)Average Success (%)
ReAct0.5815 ± 0.002219.83 ± 0.5819.44 ± 0.6931.84 ± 0.4323.71 ± 0.28
ReMe0.5825 ± 0.003422.17 ± 0.7623.41 ± 0.9132.59 ± 0.8626.06 ± 0.81
MemGen0.5734 ± 0.006018.17 ± 1.2619.64 ± 0.6035.82 ± 1.4924.54 ± 0.79
δ-mem0.6004 ± 0.003722.17 ± 1.0424.40 ± 1.1935.07 ± 0.7527.22 ± 0.98
RCMF0.6339 ± 0.001725.67 ± 0.2928.77 ± 0.3439.05 ± 1.1431.16 ± 0.58
Reviewer_66GYOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes transition-parametric memory for language agents, where each historical state-action-observation transition remains identifiable while contributing to a parametric memory. Reversible Compiled Memory Fields (RCMF) sum transition-specific key-payload outer products into a fixed-shape field. A query from the current state reads eight memory slots, which cross-attention Readers inject into a frozen Qwen3-8B policy. The Selector is pretrained and frozen, while the Writer and Readers are trained jointly. Once the modules are fixed, records can be inserted or removed through additive updates.

RCMF achieves the highest observed task-success rates among the evaluated methods on AppWorld, ALFWorld, and WebShop. Additional experiments examine ablations, memory specificity, execution cost, and field-read scaling with memory size.

Critical strengths
  1. The design reconciles identifiable records with compact runtime access in a clear and implementable way. Each record retains a separate contribution, allowing insertion and removal without changing the frozen policy or recompiling unrelated records. The paper states the limits of deletion honestly and provides a core implementation. This memory lifecycle is its main distinction from existing associative-memory mechanisms.

  2. The results also support the utility of state-conditioned memory. RCMF improves on ReAct across all three environments (Table 1), with the largest gain on AppWorld (from 32 to 48 of 168 tasks). The retrained state-independent-addressing ablation falls below ReAct on AppWorld (Table 3), despite retaining state-dependent hidden states in the Reader. This suggests that conditioning the field read on the current decision state matters beyond Reader training alone.

Critical weaknesses
  1. My main concern is that the comparative evidence does not yet establish a reliable advantage over strong alternatives. RCMF's lead over the best competing method is modest, only a few solved tasks per benchmark (two on ALFWorld), with a similarly modest gain in WebShop mean reward. These are promising point estimates, but no paired uncertainty estimates or variation across independent training runs are reported. Baseline adaptation and training details are also incomplete. The most informative missing comparison is textual replay of the same records. On beneficial training pairs, RCMF imitates a raw-text teacher receiving the Selector's chosen transition, and correct-field units also receive expert-action supervision (AppWorld, Appendix B.3). That raw-text policy is not evaluated end to end. Without this comparison, it is difficult to separate the benefit of compilation from the selected experience and supervision, or establish its practical tradeoff against textual replay.

  2. The practical benefits of reversible, growing, compiled memory are also under-validated. The record-level study (Table 4) measures action log-probability changes on a small set of related and unrelated pairs (20 each), without testing task outcomes after correcting or removing misleading records. The shuffled-field evaluations are useful held-out checks, but the same corruption types appear in training and checkpoint selection, so they provide limited independent evidence for memory specificity. Appending the transitions withheld from Writer-Reader optimization demonstrates insertion, but its behavioral benefit is not isolated. The efficiency study compares compilation with a sequential per-record read (Table 9). By my calculation from the stated dimensions, at the largest evaluated bank size the compiled contraction still requires more multiply-accumulates than a vectorized explicit read (roughly 2.0 versus 1.5 million), although operation counts alone do not establish actual latency. The first-token microbenchmark also excludes state encoding and query construction. Finally, the memory-size sweep measures latency rather than task performance. Fixed tensor shape establishes constant read complexity, but does not show that useful memory access survives increasing interference.

Questions for authors

Can you provide paired uncertainty estimates against the strongest baselines, including WebShop reward differences, and variation across independent full-system training runs?

How does RCMF compare end to end with the raw-text policy using the same Selector and eligible bank, without the offline beneficial/harmful audit at test time?

4 comments
AuthorsOpenReview ↗

Response to Reviewer 66GY (Part 1)

W1: Thank you for the constructive feedback. We address W1 in three parts: W1a Statistical Reliability of Performance Gains, W1b Baseline Adaptation and Training Details, and W1c End-to-End Comparison with Textual Replay.

W1a: The final RCMF design was completed only one month before submission, leaving insufficient time for repeated runs. We continued these experiments after submission and have now obtained three runs for all five methods (Table 1-1). All applicable training stages were rerun from scratch. RCMF achieves the highest mean on every reported metric. Its lowest observed success rate also exceeds the strongest baseline’s highest observed rate on each benchmark, although the ALFWorld margin remains small.

Table 1-1. Effectiveness across three runs (mean ± SD).

MethodWebShop RewardWebShop Success (%)AppWorld Success (%)ALFWorld Success (%)Average Success (%)
ReAct0.5815 ± 0.002219.83 ± 0.5819.44 ± 0.6931.84 ± 0.4323.71 ± 0.28
ReMe0.5825 ± 0.003422.17 ± 0.7623.41 ± 0.9132.59 ± 0.8626.06 ± 0.81
MemGen0.5734 ± 0.006018.17 ± 1.2619.64 ± 0.6035.82 ± 1.4924.54 ± 0.79
δ-mem0.6004 ± 0.003722.17 ± 1.0424.40 ± 1.1935.07 ± 0.7527.22 ± 0.98
RCMF0.6339 ± 0.001725.67 ± 0.2928.77 ± 0.3439.05 ± 1.1431.16 ± 0.58

Table 1-1 measures run-to-run variability, whereas Table 1-2 assesses uncertainty in the performance differences arising from task sampling. We compute paired 95% bootstrap confidence intervals by jointly resampling the same evaluation tasks for RCMF and the strongest original baseline.

Table 1-2. Paired uncertainty against the strongest original baselines.

MetricRCMF (reported)Strongest original baselineDifferencePaired 95% CI
AppWorld success48/168δ-mem: 41/168+4.17 pp[+0.595, +8.333] pp
ALFWorld success52/134MemGen: 50/134+1.49 pp[0.000, +3.731] pp
WebShop success51/200δ-mem: 45/200+3.00 pp[+0.500, +6.000] pp
WebShop mean reward0.6320δ-mem: 0.6017+0.0303[+0.0132, +0.0499]

The paired intervals exclude zero for AppWorld success and both WebShop metrics, supporting positive task-level differences in the reported runs. The ALFWorld CI reaches zero, leaving its two-task gain statistically uncertain. Overall, repeated runs support consistent improvements, with weaker statistical evidence on ALFWorld.

W1b: All four baselines used Qwen3-8B with the same tasks, environment, evaluator, and base prompt within each benchmark. ReAct used only the task prompt and within-episode action–observation history, without cross-task memory or training. ReMe extracted procedural guidance from successful training trajectories using frozen Qwen3-8B. At test time, Qwen3-Embedding-0.6B retrieved five entries from a fixed bank, which were reranked, rewritten, and added to the prompt. Neither model was fine-tuned, and the bank remained fixed. MemGen: Since released core code lacks runners for our benchmarks, we adapted its latent-memory generation and insertion mechanism to frozen Qwen3-8B, using interaction histories and expert next actions as training pairs. At test time, the augmented agent generated actions step by step through the shared evaluation interface. δ-mem retained online delta-rule updates, training rank-8 corrections on attention query and output projections while keeping Qwen3-8B frozen.

W1c: We evaluated the raw-text policy on all 168 AppWorld tasks using the paper’s evaluation configuration. At each step of its own rollout, it uses the same frozen Selector and eligible bank as RCMF, selects one transition by the teacher’s class-then-exemplar rule, and inserts its text into the prompt. It has no access to expert test actions or beneficial/harmful labels.

Table 1-3. AppWorld task success across memory policies.

PolicySuccess (%)
ReAct (no memory)19.05
Raw-text replay27.38
RCMF (compiled)28.57

Raw replay captures much of the improvement over ReAct, confirming the value of selected experience. 7 tasks were solved only by RCMF and 5 only by raw replay. The paired difference was +1.19 pp (95% CI [−2.98, +5.36]), so we do not establish a task-success advantage for compilation over replay. RCMF did, however, reduce median online TTFT by 35–46% on the three tested states (Table 2-4).

W2: Thank you for the insightful feedback. We address W2 in three parts: W2a Behavioral Editing and Memory Specificity, W2b Memory Insertion and Growth, and W2c Reading Efficiency and Online Latency.

AuthorsOpenReview ↗

Response to Reviewer 66GY (Part 2)

W2a: We conducted behavioral editing on 20 related and 20 unrelated AppWorld tasks using records from the 98 transitions withheld, extending Table 4 beyond log-probability probes. Related tasks are grouped by correctness of target action under Background, regardless of final task success. Each task uses the same model, decision state, and source record under 4 conditions: Background (401 records), Insert with Original (adding the original record), Replace with Misleading (replacing the original record with semantically misleading version), and Delete to Background (removing inserted record). Original and misleading records never coexist. Unlike field shuffling, replacement changes the record's semantics.

We report A (percentage of tasks for which correct action is executed at matched decision state) and T (full-task success from independent rollouts). Observed rates are in %; brackets give task-paired 95% bootstrap CIs for differences from Background (pp).

Table 2-1. Effects of memory editing on action accuracy and task success.

GroupBackgroundInsert OriginalReplace MisleadingDelete to Background
Related-Incorrect (10)A: 0 [—]; T: 20 [—]A: 80 [+50,+100]; T: 70 [+10,+90]A: 0 [0,0]; T: 10 [−30,0]A: 0 [0,0]; T: 20 [0,0]
Related-Correct (10)A: 100 [—]; T: 90 [—]A: 100 [0,0]; T: 90 [0,0]A: 20 [−100,−50]; T: 30 [−90,−30]A: 100 [0,0]; T: 90 [0,0]
Unrelated (20)A: 50 [—]; T: 60 [—]A: 50 [0,0]; T: 65 [0,+15]A: 50 [0,0]; T: 60 [−15,+15]A: 50 [0,0]; T: 60 [0,0]

Inserting original record corrects 8/10 wrong related actions while preserving correct ones. Replacing it with misleading content disrupts related decisions, whereas unrelated tasks remain largely unaffected. Deletion restores Background performance. These results provide behavioral evidence of memory specificity beyond log-probability probes.

W2b: Using the same frozen model and evaluation settings, we tested memory growth across 4 nested banks: 401-record Writer–Reader optimization pool, 499 records (adding 98 withheld transitions without weight updates), and 800 / 1600 records (adding 301 and 1101 LLM-generated synthetic variants to the 499-record bank to evaluate performance with larger banks). These variants modify entities or descriptions in training-side transitions while preserving their action semantics, structure, and provenance.

Table 2-2. Task success and retention under memory-bank growth.

Memory CountAppWorld SuccessRetained Successes from N=401
40144/16844/44
49948/16842/44
80050/16841/44
160050/16843/44

Retention counts how many of the 44 tasks solved at N=401 remain successful as the bank grows, exposing regressions despite higher overall success. All learned components remain frozen. Performance improves beyond the original Writer–Reader training-bank size, while most previously solved tasks are retained. These results provide behavioral evidence that useful memory access survives substantial synthetic interference.

W2c: We agree that vectorized explicit reading is a more appropriate baseline than sequential reading. We compared all three implementations on one H100 with identical queries, FP32 precision, and normalization.

Table 2-3. Median memory-read latency, with 1,000 measurements per condition.

Memory CountCompiled (ms)Sequential Explicit (ms)Vectorized Explicit (ms)
10.1000.1290.117
80.1020.6110.157
320.1062.2400.164
1280.1118.8100.178
2560.10717.2370.165
4990.10933.5750.171

All numerical equivalence checks passed. Despite higher estimated operation count, compiled reading is faster than vectorized explicit reading at every tested size (1.57× at N=499).

To address the omission of state encoding and query construction from the original first-token microbenchmark (Table 10 of our paper), we additionally measured complete online time to first token (TTFT), including state encoding, query construction, memory access, and model prefill. Short, Medium, and Long are the same three AppWorld prompts as Table 10 in our paper (7326, 9999, and 16985 tokens, respectively).

Table 2-4. Online TTFT (median ms; 30 measurements per policy and prompt)

PolicyShort (ms)Medium (ms)Long (ms)
No memory330464880
Raw-text replay123615862734
RCMF (compiled)6679381775

RCMF reduced TTFT by 35–46% relative to raw-text replay across the three states, while the no-memory policy remains fastest. These measurements include the online overhead omitted from the original Table 10. The corresponding end-to-end task success rates are reported in Table 1-3.

Q1: Addressed in W1a and Tables 1-1 and 1-2.

Q2: Addressed in W1c and Table 1-3.

Thanks for the complementary experiments provided by the authors, which I believe strengthen the evidence for the effectiveness of the proposed method in the paper. I've changed my overall rating to 3.

Thank you very much for considering our rebuttal and additional experiments, and for raising your overall rating to 3. We sincerely appreciate your constructive feedback. We will carefully incorporate your suggestions and ensure that our claims are appropriately supported and qualified in the revised manuscript.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.