acceptodds
Under review as a conference paper at ICLR 2027

AdaM-TTT: Adaptive Agent Memory via Test-Time Training

Abstract

Long-horizon language agents must continually adapt as tasks evolve during interaction. Agent memory supports adaptation at deployment by making relevant past experience available during interaction. Existing methods, however, operate mainly through contextual memory—retrieved trajectories or summaries placed in the current prompt—while the policy parameters remain fixed. This confines adaptation to information repeatedly supplied in the prompt, even when useful guidance should persist across subsequent decisions. We investigate whether retrieved experience can also supervise online policy adaptation, allowing memory to influence both the agent's context and its parameters. Although model parameters are conventionally viewed as long-term memory, we use a lightweight adapter as short-term parametric memory: it is updated within an episode, replaced when guidance changes, and never modifies the frozen base model. We introduce AdaM-TTT (Adaptive Agent Memory via Test-Time Training), a framework that retrieves successful experiences and distills them into a compact procedural memo relevant to the agent's current task state. The procedural memo guides action generation as contextual memory; together with a compact live-progress summary, it forms the next-token-prediction target used to update a LoRA adapter at test time. This combines immediate contextual guidance with temporary knowledge encoded in the adapter. The policy can retain this paired memory state, update it with new experience, or reset it by clearing the memo and unloading the adapter, allowing adaptation to remain responsive as the episode progresses while the base model stays frozen. On ALFWorld, AdaM-TTT achieves a 5.0% relative improvement over the strongest memory baseline, while test-time training contributes only 2.5% of estimated parameter-level compute. These results demonstrate the benefit of coupling contextual memory with short-term parametric adaptation, providing a new mechanism for continual adaptation from experience after deployment.

Primary area:
foundation or frontier models, including LLMs
Keywords:
LLM agents, test-time training

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_E3FyOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

this paper proposes AdaM-TTT, an agent memory method that uses retrieved experience in two ways. At an update, the agent retrieves 8 successful trajectories from a bank and summarizes them with the current prompt into one procedural memo. The memo goes into the action prompt, and the memo plus a separate progress summary are used as a next-token training target for a fresh LoRA adapter on the frozen base model. The policy itself decides at each step to keep, update or reset this memo and adapter pair. Evaluation is on ALFWorld (140 tasks, 3 runs) with Qwen3-4B, using the same 1,596-trajectory bank as the baselines. AdaM-TTT gets 65.2 vs 62.1 for AdaMEM, and an ablation with fixed refresh every 5 steps shows memo in context plus adapter (65.2) is better than memo in context only (58.8). TTT training is estimated as 2.51% of the method's parameter FLOPs.

Critical strengths
  1. The ablation in Table 1 shows a real within-pipeline effect. With update timing held fixed, adding the adapter on top of the in-context memo goes from 58.8 ±1.5 to 65.2 ±0.8. The variance is small so I believe this gap. This matters to me because it is the only direct evidence that a short-lived parametric update adds something beyond putting the memo in the prompt, which is the main idea of the paper (see W2 for what this comparison does and does not control).

  2. The comparison setup is controlled and the cost is reported honestly. All memory methods use the same bank, same 4B backbone, and the memo is written by the same model, so the gain cannot come from a stronger teacher. Fig. 4 reports parameter FLOPs for all baselines, and AdaM-TTT (825.1 TFLOPs/task) is actually cheaper than AdaMEM (1350.6). The authors are also clear about what their compute numbers do not cover. This makes the method look practical, which is important for a TTT method.

Critical weaknesses

The mechanism behind the adapter gain is not isolated. Each update is one short sequence, one epoch, lr 5e-5, from fresh LoRA, and the number of optimizer steps is not given. There is no placebo adapter (for example trained on unrelated or shuffled text of the same length etc), and no measure of how much the adapter changes the model. Also the TTT target contains the progress summary u_t, but u_t is never put in the prompt, so "w/o parametric memory" (58.8) removes both the adapter and all use of u_t.

The parametric-only row (46.9 ±3.2 vs 45.2 ±1.8 no memory) also suggests the adapter alone carries little of the memo. Relatedly, the KEEP/UPDATE/RESET policy, listed as a contribution, gives the same mean as fixed refresh every 5 steps (65.2 ±2.2 vs 65.2 ±0.8), and it makes only about 0.57 replacements per episode (238 over 420 episodes, line 435). This is consequential because the paper's claim is that contextual and parametric memory are complementary, and without these controls the 6.4-point gain could come from u_t or from any small weight change.

Questions for authors
  1. Can you add a placebo adapter (trained on unrelated or shuffled text of the same length, memo still in context) under the Table 1 fixed schedule? Please also state how many optimizer steps one update runs.
  2. Can you add a variant with both the memo and the progress summary u_t in context and no adapter? This is the fair control for "w/o parametric memory".
  3. The gain over AdaMEM is 65.2 ±2.2 vs 62.1 ±1.9 over 3 runs (a rough Welch test gives p around 0.14). Can you report a paired per-task test over the 140 tasks or more runs?
  4. The fixed every-5-steps variant gets the same 65.2 as the adaptive policy, with about 0.57 replacements per episode. What does the KEEP/UPDATE/RESET policy add?
2 comments

We thank the reviewer for recognizing the within-pipeline gain from the adapter, the controlled comparison under a shared bank and backbone, and the honest reporting of compute. We address each of the concerns below.

Re: Weakness 1 and Question 1. "The mechanism behind the adapter gain is not isolated. ... There is no placebo adapter ... and no measure of how much the adapter changes the model." / "Can you add a placebo adapter ... under the Table 1 fixed schedule? Please also state how many optimizer steps one update runs."

Optimizer steps. Each update is one optimizer step, which keeps test-time training efficient and avoids overfitting to a single example. With 8 steps per update, success drops from 64.3% to 60.0% (one run each).

Placebo adapter. To check whether the gain comes from the memo itself or from any small weight change, we trained the adapter on two kinds of placebo text, meaning text without useful memo content. The real memo stays in the prompt, the update schedule is the one in Table 1 of the submission (Table 2 in the revision), and each placebo matches the real text in token length.

  • Shuffled text: we randomly permute the tokens of the full training text (memo + progress summary), so it keeps its length and its tokens but loses their order.
  • Unrelated prose: 16 fixed paragraphs on astronomy and geology, written by gpt-6-astra. Each update trains on a slice of this text at a pseudo-random position. | Adapter training text | Success rate (%) ↑ | | --- | --: | | Memo + progress summary (AdaM-TTT) | 65.2 ± 0.8 | | Shuffled text | 62.9 ± 4.3 | | Unrelated prose | 62.6 ± 4.2 |

Table 3.1. ALFWorld success rate (%) by adapter training text (Qwen3-4B-Instruct-2507). We refresh the memo in all rows at step zero and every five environment steps and keep the real memo in the prompt. We report the mean ± std over three repetitions of the 140-task evaluation split.

Table 3.1 shows that the adapter trained on the real text performs best and is the most stable across runs. The placebo adapters use the same LoRA and, on one training task, change the weights by about the same amount (0.025%), but score lower. This suggests that the gain comes from what the adapter learns from the memo, not from an arbitrary weight change. The revision adds these ablations as Table 4 (Appendix D), with the construction of the placebo text in Appendix F and the optimizer-step comparison also in Appendix D.

Re: Weakness 1 and Question 2. "Also the TTT target contains the progress summary u_t, but u_t is never put in the prompt, so ... removes both the adapter and all use of u_t." / "Can you add a variant with both the memo and the progress summary u_t in context and no adapter?"

To test whether the adapter helps beyond what the prompt already provides, we ran this ablation with the memo and progress summary both in the prompt, no adapter, and the same schedule as Table 1 of the submission (Table 2 in the revision).

Method / contextSuccess rate (%) ↑
AdaM-TTT65.2 ± 0.8
Memo + progress summary, no adapter62.6 ± 1.5
Memo only, no adapter58.8 ± 1.5
Progress summary only, no adapter48.1 ± 2.5
No memory45.2 ± 1.8

Table 3.2. ALFWorld success rate (%) with and without the adapter, by what the prompt contains (Qwen3-4B-Instruct-2507). We refresh the memo in all memory methods at step zero and every five environment steps. We report the mean ± std over three repetitions of the 140-task evaluation split.

Table 3.2 shows that although putting the progress summary in the prompt helps, training the adapter on the same text gives the best result. The adapter therefore adds a benefit that the prompt alone does not provide. The revision adds this ablation as Table 4 (Appendix D).

Re: Question 3. "The gain over AdaMEM is ... over 3 runs ... Can you report a paired per-task test over the 140 tasks ...?"

To check that the gain over AdaMEM holds beyond the mean, we ran a paired per-task analysis. We report p = 0.31 from a two-sided sign-flip test over the 140 tasks, averaged over three repetitions, with AdaM-TTT ahead by 3.1 points. This difference is not significant at the 0.05 level, but a supplementary check with three further repetitions of each method confirms the same advantage for AdaM-TTT. Appendix E (Table 6) of the revision reports the paired analysis.

Re: Weakness 2 and Question 4. "The parametric-only row ... also suggests the adapter alone carries little of the memo. Relatedly, the KEEP/UPDATE/RESET policy, listed as a contribution, gives the same mean as fixed refresh every 5 steps ..." / "What does the KEEP/UPDATE/RESET policy add?"

The policy matches the fixed schedule in accuracy with far fewer training updates. The fixed schedule trains the adapter at step zero and every five environment steps. The policy trains it at step zero and afterwards only when it chooses UPDATE. Table 3.3 shows that it reaches the same mean success rate with 73.5% fewer training updates.

ScheduleSuccess rate (%) ↑Training updates ↓
KEEP/UPDATE/RESET policy (AdaM-TTT)65.2 ± 2.2658
Fixed refresh every 5 steps65.2 ± 0.82,482

Table 3.3. ALFWorld success rate (%) and number of training updates for the adaptive policy and the fixed schedule (Qwen3-4B-Instruct-2507). We report the success rate as the mean ± std over three repetitions of the 140-task evaluation split, and training updates as the total over the three repetitions (420 episodes), including the one at step zero of each episode.

The policy therefore matches the accuracy of the fixed schedule with far fewer training updates. The revision reports this in Table 3 and the split of the gain in Table 4, both in Appendix D.

Reviewer_4Jj2OpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes AdaM-TTT, a framework that combines retrieval-based agent memory with test-time training. AdaM-TTT uses retrieved successful trajectories to generate procedural memories and consolidate them into model weighst with LoRA adaptors.Experiments on ALFWorld show a 3.1% improvement over the strongest memory baseline, with ablations suggesting benefits from combining contextual and parametric memory.

Critical strengths

None

Critical weaknesses
  1. Lack of method novelty. AdaM-TTT primarily combines existing retrieval-based agent memory methods (e.g., AdaMEM) with standard LoRA-based test-time training. Therefore, the methodological contribution is limited

  2. Limited experiments. The experiments are restricted to ALFWorld with a single backbone model. More experiments are needed to evaluate the proposed method, such as experiment with other environments or models.

Questions for authors

What is the main method level contribution of AdaM-TTT compared to existing methods such as AdaMEM? The proposed approach appears to be a straightforward combination of retrieval-based memory and test-time training.

2 comments

We thank the reviewer for the careful reading and for the questions on novelty and experimental scope, which led us to add results on ALFWorld unseen, WebShop, and another backbone. We address each of the concerns below.

Re: Weakness 1 and Question 1. "Lack of method novelty. AdaM-TTT primarily combines existing retrieval-based agent memory methods ... with standard LoRA-based test-time training." / "What is the main method level contribution of AdaM-TTT compared to existing methods ...?"

While retrieval-based agent memory and test-time training offer complementary ways to adapt agent behavior, challenges remain in effectively combining them. The first challenge is experience representation, since retrieved experience must serve both as guidance in the prompt and as the training signal for an online weight update, yet raw trajectories are long and may match the current state poorly. Prior studies report that no single representation works best across settings, whether raw trajectories, distilled insights, abstract principles, or a combination (Hu et al., 2026; Tan et al., 2026; Wang et al., 2026). The second is update timing, since the useful strategy shifts as the task unfolds. Updating too frequently causes performance shift and unnecessary cost, whereas updating too infrequently means the agent does not learn enough.

AdaM-TTT addresses the first by distilling retrieved experience into a compact memo, which together with the agent's progress becomes the training target for a short-lived adapter. It addresses the second by letting the agent decide when to keep, replace, or drop the memo and adapter, and we show empirically that existing LLMs can manage this memory effectively.

Hu et al., 2026. When Continual Learning Moves to Memory: A Study of Experience Reuse in LLM Agents. arXiv preprint arXiv:2604.27003.

Tan et al., 2026. What Makes Agent Memory Useful for Reliable Unanswerable Question Handling? arXiv preprint arXiv:2608.27924.

Wang et al., 2026. More Memory, Worse Agents: Error Reproduction and Anti-Persistence in LLM Agents. arXiv preprint arXiv:2610.27406.

Re: Weakness 2. "Limited experiments. The experiments are restricted to ALFWorld with a single backbone model. ..."

To test generalization beyond ALFWorld seen and a single backbone, we added another backbone and two further evaluations, the ALFWorld unseen split and the WebShop benchmark. We evaluate with Qwen3.5-9B in an online setup, following the practice of recent agent-memory evaluations (Ouyang et al., 2026; Zhou et al., 2026; Ye et al., 2026) that run tasks as a stream while the memory grows. The setup differs from the submission as follows.

  • Memory. The submission uses a memory fixed in advance. Here the memory starts empty and grows as tasks are completed, and after each batch an LLM judge, the same model as the agent policy, decides whether each trajectory is added or discarded, so no stronger model is used and the setup remains strictly self-evolving.
  • Prompt. The memory is small at first, so retrieval is poor early on and a memo distilled from poorly matched trajectories could mislead the agent. AdaM-TTT therefore places the raw retrieved trajectories in the prompt instead of the memo, as the Synapse baseline does, while its adapter is still trained on the memo and progress summary as in the submission.
  • Task order. Tasks arrive as a stream in a shuffled order shared by all methods (five orders for ALFWorld, three for WebShop). ALFWorld processes one task at a time and WebShop, with 500 tasks, processes ten at a time. Tasks in a batch share the same memory, which is updated only between batches. Table 2.1 shows that AdaM-TTT has the highest mean success in all three settings, ahead of the strongest baseline (Synapse) by 3.4, 2.7, and 6.1 points. These results suggest that the gain is not specific to a single backbone or to ALFWorld seen, and that the method generalizes beyond them. The revision reports these results in the new Table 1 (Section 5.1).
MethodALFWorld seen success rate (%)ALFWorld unseen success rate (%)WebShop success rate (%)WebShop task score
No memory64.9 ± 2.066.4 ± 1.512.5 ± 0.723.8 ± 0.3
ReasoningBank73.0 ± 2.470.9 ± 5.717.9 ± 1.628.9 ± 1.6
Synapse75.0 ± 2.576.1 ± 5.129.4 ± 0.844.4 ± 0.8
AdaMEM72.4 ± 4.769.3 ± 5.611.8 ± 0.924.4 ± 1.3
AdaM-TTT78.4 ± 5.278.8 ± 7.035.5 ± 1.651.7 ± 2.8

Table 2.1. Online results with Qwen3.5-9B and a memory bank that grows as tasks are completed. We report the mean ± std over different task orders (five for ALFWorld seen and unseen, three for WebShop).

References

Ouyang et al., 2026. ReasoningBank: Scaling Agent Self-Evolving with Reasoning Memory. ICLR 2026.

Zhou et al., 2026. Just-in-Time Memory: Learning to Curate Task-Adaptive Memory for LLM Agents. arXiv:2609.27334.

Ye et al., 2026. On the Fragility of Self-Improving Agents: Variance, Task Order, and Underspecification. arXiv:2608.18066.

Reviewer_w7CbOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes applying procedural summaries of successful trajectories to both contextual memory and parametric memory (via test-time LoRA), so that useful guidance can better inform subsequent decision-making. On the ALFWorld benchmark, the method achieves a 5.0% relative improvement over the strongest baseline, while test-time training accounts for only 2.5% of the estimated parametric compute budget. This demonstrates the advantages of combining contextual memory with short-form parametric adaptation.

Critical strengths

Combining context grounding with test-time training (TTT) is an interesting attempt. The paper treats the test-time-trained LoRA module as short-term memory that evolves throughout the task, which stands in sharp contrast to the conventional view of parametric memory as slowly evolving long-term memory. Experiments within a limited scope show the promise of this paradigm.

Critical weaknesses

The major weakness of this work lies in insufficient experiments and analyses. The evaluated environments, baseline comparisons, and ablation studies are insufficient. The mechanism behind performance gains brought by TTT is not clearly revealed, nor is the design rationale for the TTT objective explained. Since the paper does not propose substantial technical novelty and its key difference from the primary baseline (AdaMEM) is the introduction of TTT, I argue that thorough experiments and in-depth analysis around this component should form the core of the paper.

Questions for authors
  1. Is the proposed method generalizable? Results are only reported on ALFWorld-seen, and it remains unclear whether the approach works in other scenarios. Are there experimental results on other benchmarks such as ALFWorld-unseen or WebShop?
  2. The paper mentions that aTTT and TMEM are closely related to the proposed method. Why no comparative experiments and analyses against them are provided?
  3. The original ReasoningBank uses both successful and failed attempts. This paper only employs successful trajectories in its comparisons. Could this lead to degraded performance for ReasoningBank?
  4. What benefits arise from step-0 initialization? What performance would be observed with step-0 initialization alone?
  5. Why is the fixed-update setting adopted for the ablation study?
  6. Is the w/o parametric memory variant in the ablation equivalent to AdaMEM? If yes, I have a follow-up question: why do both AdaM-TTT(fixed) and AdaM-TTT reach 65.2, while the w/o parametric memory variant is substantially lower than AdaMEM (58.8 vs 62.1)?
  7. In Figure 4, why is the compute cost of AdaMEM considerably higher than that of AdaM-TTT? Under which configuration is this comparison made?
  8. The related-work section has notable omissions in citations, including TTT (Sun et al., 2020), TTT++, Tent, CoTTA, EATA, MemGPT, and others.
3 comments

We thank the reviewer for recognizing the value of treating a test-time-trained adapter as short-term parametric memory, in contrast to the conventional view of parametric memory as slowly evolving, and for seeing the promise of this paradigm. We address each of the concerns below.

Re: Weakness 1 and Question 1. "Is the proposed method generalizable? Results are only reported on ALFWorld-seen ... Are there experimental results on other benchmarks such as ALFWorld-unseen or WebShop?"

To test generalization, we evaluate all methods with Qwen3.5-9B on ALFWorld seen, ALFWorld unseen, and WebShop. We use an online setup, following the practice of recent agent-memory evaluations (Ouyang et al., 2026; Zhou et al., 2026; Ye et al., 2026) that run tasks as a stream while the memory grows. The setup differs from the submission as follows.

  • Memory. The submission uses a memory fixed in advance. Here the memory starts empty and grows as tasks are completed, and after each batch an LLM judge, the same model as the agent policy, decides whether each trajectory is added or discarded, so no stronger model is used and the setup remains strictly self-evolving.
  • Prompt. The memory is small at first, so retrieval is poor early on and a memo distilled from poorly matched trajectories could mislead the agent. AdaM-TTT therefore places the raw retrieved trajectories in the prompt instead of the memo, as the Synapse baseline does, while its adapter is still trained on the memo and progress summary as in the submission.
  • Task order. Tasks arrive as a stream in a shuffled order shared by all methods (five orders for ALFWorld, three for WebShop). ALFWorld processes one task at a time and WebShop, with 500 tasks, processes ten at a time. Tasks in a batch share the same memory, which is updated only between batches. | Method | ALFWorld seen success rate (%) | ALFWorld unseen success rate (%) | WebShop success rate (%) | WebShop task score | | :-- | --: | --: | --: | --: | | No memory | 64.9 ± 2.0 | 66.4 ± 1.5 | 12.5 ± 0.7 | 23.8 ± 0.3 | | ReasoningBank | 73.0 ± 2.4 | 70.9 ± 5.7 | 17.9 ± 1.6 | 28.9 ± 1.6 | | Synapse | 75.0 ± 2.5 | 76.1 ± 5.1 | 29.4 ± 0.8 | 44.4 ± 0.8 | | AdaMEM | 72.4 ± 4.7 | 69.3 ± 5.6 | 11.8 ± 0.9 | 24.4 ± 1.3 | | AdaM-TTT | 78.4 ± 5.2 | 78.8 ± 7.0 | 35.5 ± 1.6 | 51.7 ± 2.8 |

Table 1.1. Online results with Qwen3.5-9B and a memory bank that grows as tasks are completed. We report the mean ± std over different task orders (five for ALFWorld seen and unseen, three for WebShop).

Table 1.1 shows that AdaM-TTT has the highest mean in all three settings, ahead of the strongest baseline (Synapse) by 3.4, 2.7, and 6.1 points. These results suggest that the gain is not specific to the submission's backbone or to ALFWorld seen, and that the method generalizes beyond them. The revision reports the online results in the new Table 1 (Section 5.1).

References

Ouyang et al., 2026. ReasoningBank: Scaling Agent Self-Evolving with Reasoning Memory. ICLR 2026.

Zhou et al., 2026. Just-in-Time Memory: Learning to Curate Task-Adaptive Memory for LLM Agents. arXiv:2609.27334.

Ye et al., 2026. On the Fragility of Self-Improving Agents: Variance, Task Order, and Underspecification. arXiv:2608.18066.

Re: Weakness 1 and Question 2. "The paper mentions that aTTT and TMEM are closely related to the proposed method. Why no comparative experiments and analyses against them are provided?"

We did not run aTTT and TMEM because they target different settings. aTTT trains a LoRA adapter during an episode on the agent's latest text, an observation, or a progress summary, and resets it for the next episode. TMEM writes question–answer pairs about the current session into fast weights when the context budget is exceeded, and targets long-context conversational memory and search. Neither is directly applicable to our setting, retrieval-based agent memory in interactive environments. aTTT learns only from the ongoing episode and cannot draw on earlier experience, and TMEM is not designed for interactive environments such as ALFWorld. AdaM-TTT instead makes parametric memory dynamic within an episode while drawing on retrieved earlier experience. We could not find a public implementation of either.

Re: Weakness 1 and Question 3. "The original ReasoningBank uses both successful and failed attempts. This paper only employs successful trajectories ... Could this lead to degraded performance for ReasoningBank?"

Our setting differs from the original ReasoningBank in the number of retrieved items and the benchmark, so we ran a variant to test the effect of adding failed attempts directly.

The ReasoningBank we evaluated retrieves eight successful attempts, whereas the added variant retrieves eight successful and eight failed attempts. As Table 1.2 shows, success dropped from 55.0% to 47.4%. This agrees with prior findings that adding failed experience to memory can hurt (Xiong et al., 2026; Tan et al., 2026), while no single best way of handling it exists (Wang et al., 2026; Lee et al., 2026). Handling failures is thus an additional design choice whose benefit varies across settings, so retrieving only successful attempts, as in our submission, does not disadvantage ReasoningBank. It is the more favorable setting for it. The revision adds this as Appendix D.

Retrieved experienceSuccess rate (%) ↑
8 successful attempts (submitted)55.0 ± 0.7
8 successful and 8 failed attempts47.4 ± 3.2

Table 1.2. ALFWorld success rate (%) of ReasoningBank by retrieved experience (Qwen3-4B-Instruct-2507, fixed bank). We report the mean ± std over three repetitions of the 140-task evaluation split.

References

Xiong et al., 2026. How Memory Management Impacts LLM Agents: An Empirical Study of Experience-Following Behavior. ACL 2026. https://aclanthology.org/2026.acl-long.27/

Wang et al., 2026. More Memory, Worse Agents: Error Reproduction and Anti-Persistence in LLM Agents. arXiv preprint arXiv:2610.27406.

Lee et al., 2026. Don't Just Trust Your Memory: Evidence-Linked Memory Reduces Memory-Induced Degradation in LLM Agents. arXiv preprint arXiv:2611.14320.

Tan et al., 2026. What Makes Agent Memory Useful for Reliable Unanswerable Question Handling? arXiv preprint arXiv:2608.27924.

Re: Weakness 1 and Question 4. "What benefits arise from step-0 initialization? What performance would be observed with step-0 initialization alone?"

To test what step-0 initialization contributes, we compare the full method with two variants and the no-memory agent. Initialization only creates the memo and adapter at the start of the episode and keeps them unchanged. No initialization starts without a memo or adapter and lets the policy create them later.

MethodMemory initializationAdaptive updatesALFWorld seen (Qwen3-4B)WebShop (Qwen3.5-9B)
AdaM-TTTYesYes (agent-decided)65.2 ± 2.239.6
Initialization onlyYesNo63.1 ± 2.236.8
No initializationNoYes (agent-decided)57.6 ± 4.315.2
No memoryNoNo45.2 ± 1.813.2

Table 1.3. Success rate (%) by initialization. We report the ALFWorld results as the mean ± std over three repetitions of the 140-task evaluation split, and the WebShop results from one run over 500 tasks.

Table 1.3 shows that initialization gives the agent useful memory from its first action, and the agent-decided updates that follow add a further gain on both benchmarks. The two are complementary. Without initialization the agent recovers only part of the benefit even though it can still update, and without later updates it falls short of the full method. The revision adds this as Appendix D (Table 5).

To see how the agent uses later updates, we analyze when it updates in the no-initialization variant, which starts without an initial memory. It triggered at least one update in 37.4% of successful and 60.0% of failed attempts, so updates occur more often in unsuccessful episodes. This suggests the agent updates selectively, when a task is hard and its guidance falls short, rather than uniformly.

Re: Weakness 1 and Question 5. "Why is the fixed-update setting adopted for the ablation study?"

We used a fixed schedule to keep the ablations fair. All variants update at the same points, so score differences reflect the component removed or changed, not update timing.

Re: Weakness 1 and Question 6. "Is the w/o parametric memory variant in the ablation equivalent to AdaMEM?"

The w/o parametric memory variant is not equivalent to AdaMEM. The update-trigger prompt differs, so agent behavior can differ. AdaM-TTT chooses among KEEP, UPDATE, and RESET, whereas AdaMEM only decides whether to keep or refresh its strategy memory. RESET lets the agent unload a memo and adapter that no longer fit as the task unfolds, an option AdaMEM does not provide. The ablation isolates the adapter within AdaM-TTT rather than reimplementing AdaMEM, so the scores (58.8 and 62.1) need not match.

Re: Question 7. "In Figure 4, why is the compute cost of AdaMEM considerably higher than that of AdaM-TTT? Under which configuration is this comparison made?"

We found a calculation error that overstated our AdaMEM estimate. With the corrected estimate in the main setting (Qwen3-4B, ALFWorld seen), AdaMEM uses 836.7 ± 34.3 TFLOPs per task, instead of the original 1,350.6, and AdaM-TTT 825.1 ± 3.2, so the two methods have comparable test-time compute. We have updated Figure 4 and its text accordingly.

AdaM-TTT reaches its accuracy at comparable compute while updating on only a small fraction of steps. The two methods spend compute differently. AdaMEM refreshes its memory more frequently, about 2.5 times per task after the first step against 0.6 updates for AdaM-TTT. AdaM-TTT has a longer prompt at each step because it also carries the adapter's update history. These effects largely offset.

Re: Question 8. "The related-work section has notable omissions in citations, including TTT (Sun et al., 2020), TTT++, Tent, CoTTA, EATA, MemGPT, and others."

To address the omissions, we expanded Section 2 of the revision into a detailed discussion of related work.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.