acceptodds
Under review as a conference paper at ICLR 2027

From Experience to Knowledge: Recursive Predictive Compression for Agent Memory

Abstract

With limited resources, agents must turn experience into reusable knowledge while retaining key details. We introduce Recursive Predictive Compression (RPC), which unifies experience internalization and generative recall through one recursive predictive representation. RPC repeatedly decomposes an experience's contribution to prediction into shared structure and local residuals using the same rule across scales. Measured predictive transfer proposes candidate scopes; later task and recall outcomes select the depth and scope of reuse and the details to retain. Compatible contributions enter a bounded world state, allowing separate records to be released while their predictive influence survives. Recall draws on internalized world knowledge, supplemented by retained key details. RPC thus connects compression, transfer, and internalization while giving each operation a distinct role. Under a fixed linear predictive interface and the stated compatibility conditions, decomposition is exact before lossy release, and absorption preserves the full predictive distribution. Evaluations on controlled decisions, hidden-rule inference, long-horizon agent memory, and visual prediction show gains in prediction and reconstruction, alongside improvements in downstream performance.

Primary area:
transfer learning, meta learning, and lifelong learning
Keywords:
Memory Internalization, Generative Recall, Predictive Compression

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_Yhf7OpenReview ↗
Rating
2(+1)
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes an approach to memory for agents that uses recursive compression to store important information efficiently. It evaluates this architecture on the symbolic alchemy task, and tests it against a range of other (reimplemented) memory architectures.

Critical strengths
  • The exact proposed memory mechanism seems novel
  • The experimental validation on is broad; I appreciated the evaluation of additional metrics on Alchemy like prediction / recall NLL as well as accuracy, the baselines compared, and the ablations.
Critical weaknesses

I have a number of concerns about the evaluation.

  • ProcGen: I suspect the policy (for both this model and all the baselines) is substantially undertrained. The original paper used up to 200m timesteps. Comparing the policies a few percent of the way into training will yield strongly biased conclusions.

  • Even on alchemy, the results seem suspiciously weak. They are not directly comparable to Wang et al.'s original results (due to changing # of trials in a given chemistry and # of potions) but the return seems far from what could optimally be achieved, and maybe comparable to a random heuristic (one of the baselines Wang et al. considered). It would be easy to score against the alchemy baselines (which were open-sourced) and it would be useful to know if the policy actually improves on the no-meta-learning random heuristic (that just learns values), and if so, by how much.

    • likewise baselines like no-memory QA on LongMem are not reported for comparison.
  • Many essential details are missing. For example, I am concerned about the amount of training on ProcGen (as noted above) but I can’t even assess whether I should be concerned about it on the other environments because (as far as I can tell) absolutely nothing is said about how much the models are trained. And many other details like what tier of LongMem / reader / etc. are not stated.

Questions for authors
  • can you clarify all the missing details about training, and e.g. how you decided to stop / when training was sufficient?
  • Can you compare to the alchemy baselines appropriately, and assess how much your model improves over the simple heuristic baselines?
  • Can you rerun experiments on procgen (and other tasks, as appropriate) with a reasonable length of training?
9 comments
AuthorsOpenReview ↗

Response to Reviewer Yhf7 (1/2)

We sincerely thank you for your time, detailed feedback, and recognition of the novelty of the mechanism and the breadth of the validation. Your careful questions on the evaluation, from whether the models are trained enough to how RPC compares with simple references, showed us that our setup needs to be stated more clearly.

Below, we address your two main concerns in turn. On whether the results are low, the Alchemy returns are high on the scale of our setting: RPC reaches 86% of the best achievable return and clearly exceeds the simple heuristic. On whether the models are trained enough, memory is written online rather than trained, and training the Procgen controller 16 to 32 times longer leaves RPC’s lead intact.

W1 & Q3. Do the Procgen results hold with a longer-trained controller?

Yes. RPC keeps the lowest visual prediction error and the highest mean return among memory methods at every training stage from 4M to 64M frames, and also on single-game Maze controllers trained for 10M frames.

First, we would like to clarify the role of the 4M-frame checkpoint. Our paper evaluates bounded online memory (64 KiB) rather than RL policy optimization, so the lightweight shared controller serves solely as a frozen common visual encoder and base policy that lets all memory mechanisms be compared on identical two-attempt trajectories within each level. What matters is therefore whether this comparison changes as the controller is trained longer.

Second, as you suggested, we reran the evaluation with longer training: shared controllers were trained up to 64M frames, with checkpoints at 16M, 32M, and 64M (4, 8, and 16 times the submitted checkpoint), and single-game Maze controllers to 10M frames (32 times the per-game frames):

ControllerNo memory MSEBest baseline MSERPC MSERPC − No memory [95% CI]No memory returnBest baseline returnRPC return
Shared 4M (Paper)0.02570.0159 (RGMem)0.0141−0.0116 [−0.0142, −0.0090]0.7090.861 (H-MEM)0.896
Shared 16M0.11110.0695 (RGMem)0.0687−0.0424 [−0.0470, −0.0379]0.7530.737 (H-MEM)0.792
Shared 32M0.21350.1349 (RGMem)0.1286−0.0849 [−0.0948, −0.0750]0.7140.702 (DeMem)0.727
Shared 64M0.36600.1929 (H-MEM)0.1787−0.1873 [−0.2083, −0.1662]0.4340.381 (DeMem)0.450
Maze 10M0.42230.2694 (RGMem)0.2487−0.1736 [−0.2180, −0.1292]2.0001.938 (Titans)2.000

Takeaway: RPC has the lowest prediction error of all methods at every stage, cutting the no-memory error by 38% to 51%, and on Maze it beats the strongest baseline, RGMem, on 85.3% of episodes. On closed-loop return, RPC also has the highest mean in every row, at or above no memory. The advantage of RPC therefore persists under 16 to 32 times more training.

As for other tasks, Procgen is the only benchmark with a trained controller, so the results on Alchemy, the controlled suite, and LongMemEval-V2 do not depend on training length. The Alchemy returns are on a smaller scale because of shorter episodes than in Wang et al., as detailed in W2 & Q2.

In the updated manuscript, Section 4.4 and Table 7 (Appendix G) report the extended results from 4M to 64M and on Maze 10M.

W2 & Q2. How much does RPC improve over the simple heuristic baselines on Alchemy?

RPC improves substantially on the random heuristic: it scores 24.2 points above the official RandomActionBot on the ten paper worlds and 19.1 points above it on 128 worlds, while every memory baseline stays within 4.7 points of the bot.

First, regarding the gap to the optimal return, our returns are on a different scale from Wang et al.: our episodes have 5 trials with 8 potions each, against 10 trials with 12 potions, and episode return grows with both. Measured against the ceiling of our own setting, an oracle that knows the true chemistry (98.8), RPC reaches 86% of the best achievable return, while the random heuristic reaches 61%.

Second, as you suggested, we ran the official RandomActionBot on the same worlds:

MethodReturn, 10 paper worldsReturn, 128 worlds
No memory56.762.1
RandomActionBot60.460.7
Best memory baseline64.0 (MemTree)65.4 (RGMem)
RPC84.679.8
Oracle with true chemistry98.897.5
RPC − RandomActionBot [95% CI]+24.2 [9.6, 38.8]+19.1 [15.9, 22.3]

Takeaway: Both paired intervals exclude zero. The gain comes from learning the hidden chemistry: RPC shares chemistry across anchors, so it predicts the outcomes of pairs it has not tried. RPC therefore improves well beyond the random heuristic.

In the updated manuscript, Section 4.2 and Table 2 (Appendix D.1) add the RandomActionBot and oracle references, and Appendix D.1 details the comparison with Wang et al.

AuthorsOpenReview ↗

Response to Reviewer Yhf7 (2/2)

W3. No-memory baseline on LongMemEval-V2

The frozen reader answers 23 of 451 questions without memory, and RPC raises this to 47, a 2.04-fold increase. As you suggested, we ran the same reader on all 451 questions with an empty memory:

MethodCorrect of 451AccuracyGain over no memory
No memory235.10%0
BM25265.76%+3
H-MEM337.32%+10
Titans419.09%+18
RPC4710.42%+24

The other baselines (DeMem, RGMem, and MemTree) answer 22 to 30 questions and are reported in Figure 8a.

Takeaway: RPC adds 24 correct answers over the reader alone, a paired gain of 5.32 percentage points [95% CI: 2.66, 8.20]. Moreover, without memory the reader returns "unknown" on 63.4% of the questions, so correct answers depend on retained content. RPC's QA count is therefore well above what the reader achieves from its own prior.

In the updated manuscript, Section 4.4 adds the no-memory reference and the paired interval, and Table 6 (Appendix G) compares RPC with every method.

W4 & Q1. What are the training amounts, stopping rules, and model settings?

Memory forms online during evaluation, so there is no training to stop. Every trained component uses a budget fixed in advance, and the table below gives the training and evaluation settings for each benchmark.

The main trained model is the Procgen controller. We train it for 5.0M frames and freeze the checkpoint with the best return on development levels (at 4.0M frames), which are disjoint from the evaluation levels, before any memory method runs. To check whether this budget is enough for comparing memory methods, we trained 16 to 32 times longer, and the conclusions hold, as shown in our response to W1 & Q3.

BenchmarkWhat is trained, and how muchStopping ruleEvaluation settings
AlchemyShared outcome prior, fitted once on 128 training worlds (5,120 transitions)Fixed data budgetOfficial symbolic environment; 5 trials, 3 stones, 8 potions, at most 20 actions per trial; 10 paper worlds and 128 additional worlds; 64 KiB cap
Controlled suiteNo shared model; memory forms online from 128 formation and 96 evidence observations per conditionFixed data budget128 test observations per condition; 25 conditions, 10 seeds; caps of 8, 16, and 32 KiB
LongMemEval-V2No shared model; the reader, encoder, and judge are frozenNo trainingMedium tier, all 451 questions; reader Qwen3.5-9B, the benchmark's official reader (4-bit, temperature 0, 256 output tokens); encoder Qwen3-Embedding-8B; judge DeepSeek-V4-Flash; 128 KiB cap, 4,096 readout tokens
ProcgenShared recurrent PPO controller, 5.0M frames over 16 gamesBest development return, then frozen320 units from 16 games; disjoint training, development, and evaluation levels; 64 KiB cap

In the updated manuscript, Table 4 (Appendix E) gives these training budgets, stopping rules, and settings.

Thanks to the authors for this thorough response.

I appreciate the comparison to more baselines, which helps to more clearly illustrate the benefit of the proposed method (e.g., on Alchemy). I also appreciate the clarification of eval tiers, etc.

On ProcGen, a few points:

  • It's pretty concerning that return is falling with more frames of training? That makes all the procgen results more suspect, since it suggests something is wrong with the training setup. The training is also still substantially shorter than the original paper.
  • It's nice to see CIs that show the proposed method significantly improves over no memory, but that's a bit of a weak baseline. The more appropriate comparison would be whether the proposed method significantly improves over the prior baselines (as is reported e.g. in Table 6 for LongMem), which I can't directly tell from the reported results, but eyeballing it looks like it would not?
  • AFAICT the paper still doesn't say whether it used the easy or hard procgen setup (the revisions only address the issues like LongMem tier that I explicitly asked about).

On LongMemV2:

  • The performance still seems suspiciously weak, with no method improving more than 5 percentage points over the no-mem baseline (by contrast, the original paper got 40-50% with Qwen3.5-9B using RAG, so there is clearly much more headroom for improvement with an effective memory system).

I have increased my rating to a 2

Reviewer_Zm3FOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

RPC for Agent Memory introduces a memory system that helps AI agents turn past experiences into reusable knowledge while keeping important details. It uses recursive compression to identify common patterns, decide which information is worth remembering, and reduce memory storage without losing useful knowledge. Experiments show that the method improves prediction, memory recall, and decision-making compared to existing approaches while using limited memory.

Critical strengths

The framework makes useful distinctions and supports them with controlled ablations, explicit assumptions, and a meaningful Alchemy return improvemen. The distinction between lossy compression, broader reuse, and prediction-preserving absorption is conceptually clear. The theory states its interface assumptions, and the experiments separate formation, validation, and evaluation. Alchemy provides the strongest downstream evidence: return improves from 64.0 to 84.6 over MemTre (p. 7). Ablations and resource accounting support several added components (p. 9).

Critical weaknesses
  • The guarantees require fixed routing and a compatible linear predictive interface. The text implementation uses token counts and retained event text and explicitly falls outside the exact coefficient-absorption guarantees. Consequently, the text results support a narrower empirical claim than the formal linear-interface results (p. 23).
  • Baselines are adapted to a shared interface and tight state caps. This supports a controlled comparison, but not an unrestricted claim of superiority over the original systems (p. 20–21).
  • Practical internalization evidence remains limited: only six absorption events occur across the two Alchemy evaluation streams. LongMemEval QA reaches 47/451 correct, and Procgen success matches H-MEM, so prediction and retention gains do not uniformly yield substantial downstream gains (p. 24–25).
Questions for authors
  1. What fraction of useful memory contributions become absorbable over longer Alchemy streams, and how much downstream benefit is specifically attributable to absorption?
  2. How closely do the constrained baseline implementations reproduce their original mechanisms and strengths? Which practical conclusions remain valid outside this shared interface?
  3. Is the QA difference of 47 versus 41 correct answers over the strongest competing count robust when accounting for questions that share histories?
4 comments
AuthorsOpenReview ↗

Response to Reviewer Zm3F (1/2)

We sincerely thank you for your time, careful reading, and recognition of the distinctions the paper draws, the controlled ablations, the explicit assumptions, the Alchemy results, and the resource accounting. We address each point below.

W1. The guarantees and the text implementation

We would like to clarify that the text implementation can meet the absorption guarantee as well: this guarantee requires only that memory is written additively, as shared knowledge plus kept details, so it holds for nonlinear models and readers. We now add this experiment below. The submission kept every event text verbatim because we also measured how much answer content reaches the reader.

Specifically, the predictor reads the sum mX(q)=W(zπ(q))+MSres(zπ(q))m_X(q)=W(z_{\pi(q)})+M_S^{\rm res}(z_{\pi(q)}), and absorption moves a correction from the second term to the first, so mX′(q)=mX(q)m_{X'}(q)=m_X(q) for every query and any readout of this sum is unchanged. As for routing, it needs to stay fixed only while two versions of memory are compared, so the guarantees keep holding as memory evolves.

On all 451 LongMemEval-V2 questions, we store text in this additive form, so the condition of the theory holds and absorption is executed exactly. The first four admitted summaries (anchor texts) move into shared state, and each event keeps only what differs from them, from which its full text is restored exactly:

QuestionsIdentical reader inputResident state (KiB)QA accuracy
451451/45178.3 → 60.4 (−22.9%)47/451, unchanged

Takeaway: The reader receives the same input on all 451 questions and gives the same answers, while the state falls by 22.9%. Absorption on text therefore carries the same guarantee as on vectors.

In the updated manuscript, Section 2.1 states that linearity is needed only in the write step, Section 3 adds the Lipschitz-readout corollary (Corollary 3.2), Table 9 (Appendix H) reports exact absorption on text, and Appendix E revises the statement on p. 23 accordingly.

W2 & Q2. How closely do the baselines reproduce the original methods, and which conclusions hold beyond the shared interface?

Each baseline reproduces the operations that define its original method, and the adaptations listed in Table 3 change mainly the representation and the budget around them. Moreover, RPC's lead holds when we vary the state cap and the frozen task model, the two components that the interface fixes.

(1) Reproducing the original mechanisms and strengths

Each baseline keeps the core operations of its original method, with the original settings where they apply (Appendix D.2, Table 3):

  • MemTree: similarity-based insertion, leaf promotion, recursive ancestor aggregation, depth-dependent routing, and retrieval across levels.
  • H-MEM: temporal and semantic consolidation with hierarchical retrieval, plus entity-graph traversal and scope-aware query planning on text.
  • RGMem: thresholded relation updates, common and specific synthesis, upward hierarchy updates, and multiscale retrieval.
  • DeMem: decision-conflict checks, witness-based slot splitting, reassignment, and split suppression at capacity.
  • Titans: associative gradient updates with a learned initialization and input-dependent learning rate, momentum, and forgetting.

The main adaptation concerns how content is represented. Where an original method asks an LLM to write summaries, numerical tasks use averages of the stored predictions, and on text every method receives the same extractive consolidation, so no method gains from a different generator. Each method also keeps its own rule for staying within the cap.

The adapted versions keep their original strengths, since each is the strongest competitor on a different task: H-MEM on Alchemy prediction and Procgen return, MemTree on Alchemy return and controlled-suite decisions, RGMem on Procgen prediction, and Titans on LongMemEval-V2 QA.

(2) Conclusions beyond the shared interface

  • State cap (Appendix F). RPC at 8 KiB beats every baseline at 32 KiB, four times the memory, on all four main controlled-suite measures.
  • Frozen task model (new, Table 7 in Appendix G). With Procgen controllers trained from 4M up to 64M frames, and with 10M-frame Maze controllers, RPC keeps the lowest recall error and the highest mean return among memory methods in every setting.

The results on RPC's own mechanisms, namely exact absorption, the component ablations, and bounded state over long streams (Sections 4.2 and 4.5, and Appendix H for the new text case), compare RPC only with itself, so they hold whatever the baselines.

In the updated manuscript, Section 4.1 and Appendix D.2 state the scope of the comparison, Table 3 lists the retained operations, adaptations, and cap accounting for each baseline, and Table 7 (Appendix G) reports the controller variations.

AuthorsOpenReview ↗

Response to Reviewer Zm3F (2/2)

W3, Q1 & Q3. Practical evidence: absorption on longer streams, robustness of the QA difference, and Procgen success

We have extended the evidence on all three points. (1) Absorption grows with the stream and saves memory without loss, which turns into accuracy when memory is tight. (2) On LongMemEval-V2, RPC answers twice as many questions as no memory, and before the reader it retains more of each answer than any other method. (3) On Procgen, RPC leads in return and visual prediction, which are finer than binary success.

1(Q1). Absorption on longer Alchemy streams

The six absorptions in the paper come from only 10 worlds with 5 trials each. We extended Alchemy to 80 trials over 32 worlds. Each cell gives the fixed intervention stream, then the agent's own closed-loop episodes:

TrialsAbsorptions per worldShare of updates absorbedBytes released per world
50.63 / 0.418.1% / 6.5%90 / 59
202.41 / 1.636.7% / 6.1%347 / 234
808.78 / 5.195.7% / 5.4%1,265 / 747

Takeaway: Absorption grows in proportion to the stream, reaching 447 absorptions at 80 trials rather than six. Each one preserves predictions to float32 precision, as Proposition C.4 predicts, while return per trial rises from 17.74 to 20.77 and resident state grows by only 12.4%. Absorption keeps working on long streams and frees memory without loss.

As for how much of the downstream gain comes from absorption, absorption’s benefit is the memory it saves without loss: it moves knowledge into the world state and releases the records, leaving every prediction unchanged. Empirically:

  • With ample memory, absorption changes no result but saves space. Return is 20.77 per trial at 80 trials with and without it, so the rise from 17.74 comes from the longer stream, while controlled-suite state falls from 20.75 to 17.27 KiB.
  • When memory is tight, the saved space becomes accuracy. Full RPC keeps fewer records yet has significantly lower future-prediction error at every tight cap, and in closed-loop Alchemy it more than halves the records evicted for capacity.

RPC's larger gains, such as the Alchemy return of 84.6 against 64.0, come from sharing and reuse.

2(Q3). Robustness of the LongMemEval-V2 QA count

RPC's QA lead is robust against no memory and five of the six baselines. RPC answers 47 of 451 questions, twice the 23 without memory. BM25, DeMem, RGMem, MemTree, and H-MEM answer 22 to 33, and the 95% interval of RPC's lead over each of them lies above zero, with questions that share a history resampled together.

As for Titans, its QA count of 41 is close to RPC's 47, because the shared official reader, Qwen3.5-9B, answers only about one question in ten even with memory and so caps every method. Before the reader, RPC retains 64% more answer tokens (0.519 vs. 0.317) and 28% more exact answers (0.305 vs. 0.239), both significant. These finer measures reflect what the memory itself keeps, and on them RPC leads every method, Titans included.

3. Procgen success

Success ties because it sits near its floor. The frozen controller completes only about 6% of levels, and success counts only fully solved levels, so memory rarely changes the outcome.

On the finer measures, RPC leads:

  • Return and visual prediction. Its attempt-two return, which credits partial progress, is the highest (0.896 vs. 0.861 for H-MEM), and its next-latent MSE is 11% below the best baseline.
  • Beyond this controller. With controllers trained from 4M up to 64M frames, RPC keeps the lowest recall error and the highest mean return at every stage (Table 7 in Appendix G).

Beyond this task, where decisions are scored more finely, RPC's gain is large: Alchemy return rises from 64.0 with the best baseline to 84.6, and RPC has the highest decision accuracy and utility in the controlled suite.

In the updated manuscript, Appendix H adds the 80-trial absorption records (Table 10) and the tight-cap comparison, and Table 6 (Appendix G) adds the paired tests against every baseline.

Thank you for the detailed responses and the additional experiments. I appreciate the effort the authors have put into addressing the concerns raised during the discussion.

The new results help clarify several points from my original review. In particular, the LongMemEval-V2 absorption experiment, the longer Alchemy tests, and the additional Procgen experiments provide stronger support for the proposed method. I also appreciate the clearer explanation of the baselines and how the experiments were setup.

I still think some limitations are worth mentioning. While the improvements in memory efficiency and prediction are promising, they do not always lead to clear improvements in actual task performance. This is especially true for the low QA accuracy on LongMemEval-V2 and the less clear results for Procgen return, which remains a concern.

Overall, I think the authors have addressed most of my concerns and the additional experiments has made the paper stronger. I find the distinction between compression, reuse, and absorption interesting and think it is a useful contribution.

I will therefore keep my current rating. I would just encourage the authors to be careful with broader claims about practical benefits where the results are still limited.

AuthorsOpenReview ↗

Appreciation for the Reviewer’s Feedback

Thank you once again for your thoughtful and constructive feedback, and for recognizing the distinction between compression, reuse, and absorption as an interesting and useful contribution.

We also appreciate your positive assessment of the additional experiments and clarifications. We acknowledge that the gains on some metrics are less pronounced and will reflect this clearly in the revision. Thank you for the time and care you have devoted to helping us improve our manuscript.

Best regards,
Authors

Reviewer_3B26OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes Recursive Predictive Compression(RPC), a new momery framework for LLM agent. It separates predictive corrections into common patterns and specific details, learns from new experience to decide when past lessons apply and combines useful information into long-term memory. Experiments on synthetic tasks, Symbolic Alchemy, LongMemEval-V2, and Procgen shows its effectiveness.

Critical strengths
  1. The paper analyses the distinction between compression, reuse, and absorption, which is promising. The paper explicitly distinguishes what to retain, when to reuse past experience, and how to combine it into long-term memory.
  2. The paper distinguishes difference between removing a record and losing its knowledge. Agents do not needs to save a complete record once its core information has been involved in shared memory. This offers a appealing new view of forgetting for long-term memory.
Critical weaknesses
  1. Mismatch between theory and implementation. Although mathematical analysis part is elegant, it assumes a vector representation of memory with a linear readout. However, these guarantees may not established when the method is complex, such as LLM agent memory. LongMemEval-V2 uses token counts and retrieved text, and do not directly implement the proposed absorption. Thus, method seems to be inspired by the theory rather than an implementation of it.

  2. Method complexity and insufficient ablations. RPC includes many components, such as grouping, scope selection, calibration and gates, which makes the reviewers hard to find out which components lead to the final gains. The paper includes several ablations, but some discuss and remove multiple mechanisms at once, making the conclusions less convincing.

Questions for authors
  1. What are the main advantages of RPC compared with tree-based memory search methods?
  2. What distinguishes agent memory tasks from other tasks in the paper? Are there specific properties of agent memory that could be used to guide memory search?
5 comments
AuthorsOpenReview ↗

Response to Reviewer 3B26 (1/3)

We sincerely thank you for your time, constructive feedback, and recognition of the distinction among compression, reuse, and absorption, and of the difference between removing a record and losing knowledge. We address each point below.

W1. Relation between the theory and the implementation

The implementation follows the theory: the only linearity the guarantees use is in how memory is written, which is our own design choice rather than an assumption on the task or the model. The guarantees therefore hold with nonlinear models and readouts, and absorption preserves prediction exactly on both vectors and text, including LongMemEval-V2.

(1) The guarantees hold without a linear readout

Memory meets the predictor in two steps: writing, which stores a correction, and reading, which turns stored memory into a prediction. Only writing is linear, and only in the stored values, not in the inputs: each record adds its correction to memory, and the predictor reads the sum of shared knowledge and kept details,

mX(q)=W(zπ(q))+MSres(zπ(q))m_X(q)=W(z_{\pi(q)})+M_S^{\rm res}(z_{\pi(q)}).

As in kernel methods, the features can be nonlinear and only their combination is additive. The base model, the task distribution, the decision rule, and the encoder can all be nonlinear (Section 2.1).

On this basis, each guarantee holds without a linear readout:

  • Absorption, which is why we make writing additive. Moving a record's correction from the details into shared knowledge, W′=W+ΨζcW'=W+\Psi_\zeta c and M′=MSres−ΨζcM'=M_S^{\rm res}-\Psi_\zeta c, leaves the sum unchanged, mX′(q)=mX(q)m_{X'}(q)=m_X(q) for every query. The prediction is therefore unchanged for any readout of this sum, and the record can be released with its effect intact.

  • Compression risk (Theorem 3.1). The proof uses linearity in one step only, to bound the change in predicted parameters by the change in the memory field. Any bb-Lipschitz readout gives the same step (Corollary 3.2), ∥δ(q)∥≤b∥e(zπ(q))∥\Vert\delta(q)\Vert\le b\Vert e(z_{\pi(q)})\Vert, so the bound R(X′)−R(X)≤εXbγD+Lb2γ2D2R(X')-R(X)\le\varepsilon_X b\sqrt{\gamma}D+\tfrac{Lb^2\gamma}{2}D^2 holds unchanged.

  • Decomposition (Proposition C.1) is algebraic, and reuse (Theorem 3.3) and scope selection (Proposition C.3) need only bounded loss differences and independent future observations.

(2) Absorption is now executed exactly on LongMemEval-V2

LongMemEval-V2 runs the same structure on text: shared token counts and anchor texts play the role of shared knowledge WW, and kept event texts play the role of the details MresM^{\rm res}. As you pointed out, the submitted text version kept every event text, so absorption was not executed. We kept them because we also wanted to measure how much answer content reaches the reader.

We now execute absorption on all 451 questions: the four anchor texts move into shared state and are released from their records, and each later event keeps only what differs from them, from which its full text is restored exactly. For text, the counterpart of mX′(q)=mX(q)m_{X'}(q)=m_X(q) is that the reader receives exactly the same input, so the frozen LLM gives exactly the same output, however nonlinear it is:

QuestionsIdentical reader inputResident state (KiB)QA accuracy
451451/45178.3 → 60.4 (−22.9%)47/451, unchanged

Takeaway: The reader input is identical on all 451 questions while the state falls by 22.9%. Absorption on text therefore carries the same guarantee as on vectors.

In the updated manuscript, Section 2.1 states that linearity is needed only in the write step, Section 3 adds the Lipschitz-readout form of Theorem 3.1 as Corollary 3.2, with the proof in Appendix C.1, and Section 4.4 and Table 9 (Appendix H) add the exact text absorption on LongMemEval-V2.

AuthorsOpenReview ↗

Response to Reviewer 3B26 (2/3)

W2. Method complexity and single-factor ablations

In fact, RPC consists of only three operations, compression, reuse, and absorption, and every other component is a smaller step within one of them. Each has its own single-factor ablation, and to address your concern, we have now split the one joint ablation into its three parts.

(1) Regarding complexity, each operation does one job on the same split of an experience into a shared part and local details:

  • Compression releases details that later evidence shows are not needed.
  • Reuse decides how widely the shared part applies.
  • Absorption moves compatible parts into the world state, so a record can be released while its effect on prediction remains.

The components you listed are steps of the reuse operation: grouping proposes candidate scopes, scope selection picks one, and the evidence gate accepts it only if later outcomes improve. Task calibration is not part of RPC itself: it is shared identically by every method, including the baselines.

(2) Regarding ablations, the paper already removes scope selection, the evidence gate, local details, and absorption one at a time, and each removal significantly worsens the metric it targets. The joint ablation was no hierarchical sharing, which removes grouping, cross-node sharing, and ancestor expansion together. We have now removed each of the three alone:

Removed aloneMetricFull RPCAblatedPaired difference [95% CI]
GroupingFuture MSE1.2261.234−0.008 [−0.012, −0.003]
Cross-node sharingFuture MSE1.2261.242−0.016 [−0.021, −0.010]
Weighted utility0.2230.187+0.036 [+0.021, +0.052]
Ancestor expansionFuture MSE1.2261.241−0.014 [−0.019, −0.010]
Weighted utility0.2230.195+0.029 [+0.018, +0.040]
Cross-node sharing (Alchemy)Future-recall NLL0.9511.327−0.376 [−0.537, −0.216]

Takeaway: Removing any one of the three alone significantly worsens its target metric. Moreover, cross-node sharing and ancestor expansion also improve decisions, and in Alchemy cross-node sharing accounts for most of the joint effect on recall (1.327 vs. 1.548 when all three are removed). Every component of RPC therefore has its own measured contribution.

In the updated manuscript, Section 2 opens with this three-operation overview, and Tables 13 and 14 (Appendix J) add the three single-factor sharing ablations on both benchmarks with all metrics.

AuthorsOpenReview ↗

Response to Reviewer 3B26 (3/3)

Q1. What are the main advantages of RPC over tree-based memory search methods?

The main advantage is that RPC turns experience into knowledge, whereas tree-based methods such as MemTree and H-MEM store records and search them. A tree groups records and their summaries by similarity, and recall finds the closest ones. RPC instead keeps what each experience contributes to prediction: the shared part is integrated into a world state, much as people distill experiences into a general understanding of the world [1], and recall reconstructs from this state plus the details that still matter [2].

This difference shows up in each of RPC's three operations:

  • Compression. A tree summarizes children into a parent with no statement of what is lost. RPC separates details from the shared part, and releasing them has the risk bound of Theorem 3.1.
  • Reuse. A tree places a new item under its most similar node at insertion. RPC deploys a broader scope only if later outcomes show a positive gain G=pFE[ℓkeep−ℓjump∣F]G=p_F\mathbb{E}[\ell_{\rm keep}-\ell_{\rm jump}\mid F] (Theorem 3.3), so where knowledge applies is learned from evidence rather than fixed by similarity.
  • Absorption. A tree either keeps or deletes a node. RPC absorbs a compatible record into the world state and releases it with predictions unchanged (Proposition C.4), so the knowledge outlives the record.

Empirically, under matched memory caps on Alchemy, RPC lowers future-recall NLL from H-MEM's 1.234 to 0.951 and raises return from MemTree's 64.0 to 84.6, a paired gain of 20.6 [95% CI: 6.2, 35.0].

In the updated manuscript, Appendix A adds this comparison with tree-based memory.

[1] McClelland et al., "Why there are complementary learning systems in the hippocampus and neocortex: Insights from the successes and failures of connectionist models of learning and memory," Psychological Review, 1995.

[2] Spens and Burgess, "A generative model of memory construction and consolidation," Nature Human Behaviour, 2024.

Q2. What distinguishes agent memory tasks, and which of their properties can guide memory search?

What sets agent memory apart is that the agent both recalls its experience and draws on it to predict and decide. Every task in the paper is an agent memory task in this sense, so we evaluate each one on both recall and the predictions and decisions that memory supports. RPC improves both: for recall, it raises LongMemEval-V2 answer-token reconstruction from 0.323 (BM25) to 0.519, and for decisions, it raises Alchemy return from 64.0 to 84.6 (Q1).

Regarding memory search, the key property is that an agent's experiences come from one persistent world. A new query therefore mostly needs regularities shared across many experiences rather than one particular record, much as people recall a specific scene through their general understanding of the world. Alchemy's chemistry holds across trials, and a Procgen level holds across attempts.

This suggests searching memory from general to specific, which is what RPC does:

  • Recall starts from the world state, which holds these regularities, and adds stored details only as far as the query needs. As a result, memory can answer queries that no stored record covers. On a new stone pairing in Alchemy, the world state alone gives a 3.99-percentage-point probability gain over no memory, close to full RPC's 4.38.
  • How far to go is set by a second property: the agent later observes outcomes, so the level that better predicts them is chosen, whereas similarity-based search, including the tree-based methods in Q1, looks for the stored records closest to the query. In the multiscale task, RPC also chooses the matching depth for every query.

In the updated manuscript, Section 1 states these two aspects of evaluation, and Appendix A discusses how a persistent world guides memory search.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Response to AC and Reviewers: Summary of Revisions

Dear Area Chair and Reviewers,

We sincerely thank all reviewers for their careful and constructive engagement. In particular, we thank them for recognizing the novelty of the proposed memory mechanism (Yhf7), that the distinction between lossy compression, broader reuse, and prediction-preserving absorption "is conceptually clear" (Zm3F), and that separating removing a record from losing its knowledge offers an "appealing new view of forgetting for long-term memory" (3B26). We also appreciate their recognition of the broad evaluation, explicit assumptions, and controlled ablations.

Core contribution. RPC turns experience into reusable knowledge under a fixed memory budget.

  • Reusable knowledge shared across experiences in one persistent world is internalized into a world state, which becomes the agent's own understanding of how the world works.

  • Only the surprising details that this understanding cannot yet predict are kept as residuals, and as understanding deepens, these details are absorbed and released.

  • Recall is thus also a form of generation rather than a search for the most similar record.

All of this is realized through one recursive mechanism with three operations: compression, reuse, and absorption.

Main revisions.

  1. Theory and implementation (3B26, Zm3F). Section 2.1 clarifies that only the readout interface is linear, while the task distribution, decision link, and base model may be nonlinear, and Corollary 3.2 extends the guarantee to Lipschitz readouts. Executed on text, absorption keeps all 451 LongMemEval-V2 reader inputs identical while resident state falls from 78.3 to 60.4 KiB (Table 9).

  2. Ablations, absorption, and baselines (3B26, Zm3F). Single-factor ablations on the controlled suite and Alchemy (Tables 13 and 14), 447 absorption events on 80-trial Alchemy streams that preserve predictions to float32 precision (Table 10), a comparison with tree-based memory (Appendix A), and the retained mechanisms of each baseline (Table 3).

  3. Benchmark alignment and longer training (Yhf7). The official RandomActionBot, a true-chemistry oracle, and the canonical 10-trial, 12-potion Alchemy configuration (Table 2); training budgets (Table 4) and complete benchmark details; and Procgen controllers trained longer, with shared controllers from 4M to 64M frames and single-game controllers up to 50M frames on five games (Table 7).

On the two non-significant results, the final concern shared by the reviewers. The reviewers noted two downstream results where RPC's advantage is less pronounced: Procgen closed-loop performance, where RPC is statistically tied with no memory, and LongMemEval-V2 QA accuracy, where its difference from Titans (47 vs. 41 of 451) is not significant. Following the reviewers' suggestion, we narrow our claims about these two results. We also believe that reporting where the gains are clear and where they are not gives a more faithful picture of the method than uniformly positive results. Across the remaining results, RPC's gains are clear and significant. In canonical Alchemy, RPC reaches 224.9 versus 169.8 for the strongest memory baseline. On LongMemEval-V2, it answers twice as many questions as no memory and retains more answer content before the shared reader than every method, Titans included (0.519 vs. 0.317 answer-token reconstruction). On Procgen, it has significantly lower prediction error than every competing memory method on single-game controllers.

We sincerely thank all reviewers again for their insightful comments and active engagement throughout the discussion, which have strengthened the paper. All revisions are incorporated into the revised manuscript and marked in blue. We are also grateful to the Area Chair for overseeing the review process and for considering our responses.

Sincerely,

Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.