- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
this paper proposes AdaM-TTT, an agent memory method that uses retrieved experience in two ways. At an update, the agent retrieves 8 successful trajectories from a bank and summarizes them with the current prompt into one procedural memo. The memo goes into the action prompt, and the memo plus a separate progress summary are used as a next-token training target for a fresh LoRA adapter on the frozen base model. The policy itself decides at each step to keep, update or reset this memo and adapter pair. Evaluation is on ALFWorld (140 tasks, 3 runs) with Qwen3-4B, using the same 1,596-trajectory bank as the baselines. AdaM-TTT gets 65.2 vs 62.1 for AdaMEM, and an ablation with fixed refresh every 5 steps shows memo in context plus adapter (65.2) is better than memo in context only (58.8). TTT training is estimated as 2.51% of the method's parameter FLOPs.
Critical strengths
-
The ablation in Table 1 shows a real within-pipeline effect. With update timing held fixed, adding the adapter on top of the in-context memo goes from 58.8 ±1.5 to 65.2 ±0.8. The variance is small so I believe this gap. This matters to me because it is the only direct evidence that a short-lived parametric update adds something beyond putting the memo in the prompt, which is the main idea of the paper (see W2 for what this comparison does and does not control).
-
The comparison setup is controlled and the cost is reported honestly. All memory methods use the same bank, same 4B backbone, and the memo is written by the same model, so the gain cannot come from a stronger teacher. Fig. 4 reports parameter FLOPs for all baselines, and AdaM-TTT (825.1 TFLOPs/task) is actually cheaper than AdaMEM (1350.6). The authors are also clear about what their compute numbers do not cover. This makes the method look practical, which is important for a TTT method.
Critical weaknesses
The mechanism behind the adapter gain is not isolated. Each update is one short sequence, one epoch, lr 5e-5, from fresh LoRA, and the number of optimizer steps is not given. There is no placebo adapter (for example trained on unrelated or shuffled text of the same length etc), and no measure of how much the adapter changes the model. Also the TTT target contains the progress summary u_t, but u_t is never put in the prompt, so "w/o parametric memory" (58.8) removes both the adapter and all use of u_t.
The parametric-only row (46.9 ±3.2 vs 45.2 ±1.8 no memory) also suggests the adapter alone carries little of the memo. Relatedly, the KEEP/UPDATE/RESET policy, listed as a contribution, gives the same mean as fixed refresh every 5 steps (65.2 ±2.2 vs 65.2 ±0.8), and it makes only about 0.57 replacements per episode (238 over 420 episodes, line 435). This is consequential because the paper's claim is that contextual and parametric memory are complementary, and without these controls the 6.4-point gain could come from u_t or from any small weight change.
Questions for authors
- Can you add a placebo adapter (trained on unrelated or shuffled text of the same length, memo still in context) under the Table 1 fixed schedule? Please also state how many optimizer steps one update runs.
- Can you add a variant with both the memo and the progress summary u_t in context and no adapter? This is the fair control for "w/o parametric memory".
- The gain over AdaMEM is 65.2 ±2.2 vs 62.1 ±1.9 over 3 runs (a rough Welch test gives p around 0.14). Can you report a paired per-task test over the 140 tasks or more runs?
- The fixed every-5-steps variant gets the same 65.2 as the adaptive policy, with about 0.57 replacements per episode. What does the KEEP/UPDATE/RESET policy add?
2 comments
Re: Question 3. "The gain over AdaMEM is ... over 3 runs ... Can you report a paired per-task test over the 140 tasks ...?"
To check that the gain over AdaMEM holds beyond the mean, we ran a paired per-task analysis. We report p = 0.31 from a two-sided sign-flip test over the 140 tasks, averaged over three repetitions, with AdaM-TTT ahead by 3.1 points. This difference is not significant at the 0.05 level, but a supplementary check with three further repetitions of each method confirms the same advantage for AdaM-TTT. Appendix E (Table 6) of the revision reports the paired analysis.
Re: Weakness 2 and Question 4. "The parametric-only row ... also suggests the adapter alone carries little of the memo. Relatedly, the KEEP/UPDATE/RESET policy, listed as a contribution, gives the same mean as fixed refresh every 5 steps ..." / "What does the KEEP/UPDATE/RESET policy add?"
The policy matches the fixed schedule in accuracy with far fewer training updates. The fixed schedule trains the adapter at step zero and every five environment steps. The policy trains it at step zero and afterwards only when it chooses UPDATE. Table 3.3 shows that it reaches the same mean success rate with 73.5% fewer training updates.
| Schedule | Success rate (%) ↑ | Training updates ↓ |
|---|---|---|
| KEEP/UPDATE/RESET policy (AdaM-TTT) | 65.2 ± 2.2 | 658 |
| Fixed refresh every 5 steps | 65.2 ± 0.8 | 2,482 |
Table 3.3. ALFWorld success rate (%) and number of training updates for the adaptive policy and the fixed schedule (Qwen3-4B-Instruct-2507). We report the success rate as the mean ± std over three repetitions of the 140-task evaluation split, and training updates as the total over the three repetitions (420 episodes), including the one at step zero of each episode.
The policy therefore matches the accuracy of the fixed schedule with far fewer training updates. The revision reports this in Table 3 and the split of the gain in Table 4, both in Appendix D.
We thank the reviewer for recognizing the within-pipeline gain from the adapter, the controlled comparison under a shared bank and backbone, and the honest reporting of compute. We address each of the concerns below.
Re: Weakness 1 and Question 1. "The mechanism behind the adapter gain is not isolated. ... There is no placebo adapter ... and no measure of how much the adapter changes the model." / "Can you add a placebo adapter ... under the Table 1 fixed schedule? Please also state how many optimizer steps one update runs."
Optimizer steps. Each update is one optimizer step, which keeps test-time training efficient and avoids overfitting to a single example. With 8 steps per update, success drops from 64.3% to 60.0% (one run each).
Placebo adapter. To check whether the gain comes from the memo itself or from any small weight change, we trained the adapter on two kinds of placebo text, meaning text without useful memo content. The real memo stays in the prompt, the update schedule is the one in Table 1 of the submission (Table 2 in the revision), and each placebo matches the real text in token length.
Table 3.1. ALFWorld success rate (%) by adapter training text (Qwen3-4B-Instruct-2507). We refresh the memo in all rows at step zero and every five environment steps and keep the real memo in the prompt. We report the mean ± std over three repetitions of the 140-task evaluation split.
Table 3.1 shows that the adapter trained on the real text performs best and is the most stable across runs. The placebo adapters use the same LoRA and, on one training task, change the weights by about the same amount (0.025%), but score lower. This suggests that the gain comes from what the adapter learns from the memo, not from an arbitrary weight change. The revision adds these ablations as Table 4 (Appendix D), with the construction of the placebo text in Appendix F and the optimizer-step comparison also in Appendix D.
Re: Weakness 1 and Question 2. "Also the TTT target contains the progress summary u_t, but u_t is never put in the prompt, so ... removes both the adapter and all use of u_t." / "Can you add a variant with both the memo and the progress summary u_t in context and no adapter?"
To test whether the adapter helps beyond what the prompt already provides, we ran this ablation with the memo and progress summary both in the prompt, no adapter, and the same schedule as Table 1 of the submission (Table 2 in the revision).
Table 3.2. ALFWorld success rate (%) with and without the adapter, by what the prompt contains (Qwen3-4B-Instruct-2507). We refresh the memo in all memory methods at step zero and every five environment steps. We report the mean ± std over three repetitions of the 140-task evaluation split.
Table 3.2 shows that although putting the progress summary in the prompt helps, training the adapter on the same text gives the best result. The adapter therefore adds a benefit that the prompt alone does not provide. The revision adds this ablation as Table 4 (Appendix D).