acceptodds
Under review as a conference paper at ICLR 2027

T²Mem: Learning Test-Time Memory for Robotics

15 views

Abstract

Memory-dependent robotic manipulation requires policies to use information that is no longer available in the current observation. Retaining history alone is insufficient: memory must preserve information that supports future actions. We introduce T²MEM, a framework that develops this capability within a pretrained vision-language-action policy, without external reasoning models or memory-specific annotations. T²MEM uses test-time training to encode observation history into compact fast weights through online self-supervised updates, avoiding repeated processing of the full history. An observation-grounded interface extracts vision-language information for memory formation and supplies retrieved context to the action expert. Action supervision shapes what the memory learns to retain and use, while alternating memory–policy learning gives each component a fixed counterpart during optimization. Across 16 RoboMME tasks, T²MEM improves average success from 17.93% to 56.83% over the memory-free base policy and outperforms the recurrent-memory methods reported in the benchmark, while controlled profiling indicates at least 3x inference speedup over explicit methods.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Robotic Manipulation, Vision-Language-Action Models, Test-Time Training, Self-Supervised Learning, Memory

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_LW6gOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper adds a fast-weight TTT memory to π0.5. Learned query tokens cross-attend to VLM features to form a summary U_t. U_t is projected to K/V for a self-supervised associative write and to Q for a read. The gated readout is given to the action expert as extra keys and values. Training alternates between two phases: memory slow parameters are trained with the policy frozen, then the policy is trained with memory frozen. A heuristic scale β_t modulates write strength. On the 16 RoboMME tasks, average success rises from 17.9% (π0.5) to 56.8%. RoboMME's own TTT/RMT memory variants reach about 18–22%. Intervention experiments (disabling writes, swapping memory between episodes, restricting the write window) show that actions depend on what is stored in memory.

Critical strengths
  1. The empirical gain is large. On the same backbone and with the same memory family (fast-weight TTT), the method gains about 35 points over the recurrent-memory baselines. It does this with a constant-size state and no external VLM calls.
  2. Sec. 4.4 is the strongest part of the paper. Using conflicting memory drops VideoUnmask from 300/300 to 1/300. Empty memory falls to chance. Pre- vs. post-occlusion writes show where the useful information is stored. Few memory-VLA papers test whether the memory content is actually used.
Critical weaknesses
  1. The main gain is not attributed. All baseline numbers in Table 1 are copied from the RoboMME paper. Training steps, compute (32 H200 × 48 h here) and the multi-task setup are therefore not controlled. RoboMME's TTT variants share the backbone and memory type and score about 22%. The paper never shows, on the benchmark, how the 35-point gap splits between the interface, alternation, β_t and the training budget. The needed experiment: rerun RoboMME's TTT-{Context, Modul, Expert} under the authors' recipe and budget, then add components one at a time on all 16 tasks.
  2. The ablations are too thin for the claims they support. They use two tasks and single-task checkpoints, with no variance reported.
  • The RoboTTT-style variant (30/26) is at the no-memory level (23/28). RoboTTT reports that memory capability appears during its frozen-backbone memory training stage. If the reimplementation skips that stage, this baseline is a straw man.
  • "Non-alternating" means joint training from the start, and it also lands at the no-memory level. The baseline that would justify repeated alternation is a single round: train memory with the policy frozen, then train jointly. The number of alternation rounds is also not reported.
  • Adaptive writing is evaluated only by the NMSE of early K/V bindings. Damping later writes reduces interference with early bindings almost by construction. There is no task-success result, and no comparison with existing forgetting or surprise gating (Titans, gated delta rules). As presented, β_t is an unvalidated heuristic.
  1. The swap tasks expose the limit of the memory claim. VideoUnmaskSwap and ButtonUnmaskSwap are the weakest tasks, at about 20%. These tasks require updating a stored binding, which is exactly where an adaptive associative write should help. L362–367 blame π0.5's temporal reasoning, but this is not tested. The evidence fits a memory that stores a cue seen once and retrieves it later, i.e., mainly the Permanence/Reference regime.
  2. Evaluation scope is narrow. There is one simulated benchmark, no real-robot experiment and no second memory benchmark (e.g., MemoryBench). The efficiency comparison only covers FrameSamp and MemER. The relevant comparison is against constant-cost recurrent memory (TTT, RMT). Being faster than explicit-history methods is expected.
Questions for authors
  1. What compute and how many steps did the RoboMME baselines use, compared with yours? Can you rerun the TTT baselines under your training recipe?
  2. Does the RoboTTT-style variant include RoboTTT's frozen-backbone memory training stage? Was it trained with the same alternating schedule?
  3. What is task success with β_t = 1 on the full benchmark?
  4. How many alternation rounds are used, and how sensitive are results to this number? How does one round compare with repeated alternation?
  5. During training, how are gradients propagated through the fast-weight writes (full BPTT or truncated), and over what sequence length?
9 comments
AuthorsOpenReview ↗

Response to Reviewer LW6g (Part 1/5)

We sincerely thank the reviewer for the thorough and constructive review. We are glad that the reviewer finds the empirical gain large: on the same backbone and with the same fast-weight TTT memory family, T²Mem gains about 35 points over the recurrent-memory baselines, with a constant-size state and no external VLM calls. We also appreciate that the reviewer considers Sec. 4.4 the strongest part of the paper, where conflicting memory drops VideoUnmask from 300/300 to 1/300, empty memory falls to chance, and pre- versus post-occlusion writes show where the useful information is stored. Below we respond to each weakness and question in turn.

Weakness 1 + Question 1

We thank the reviewer for raising this important question. We first answer Question 1 by comparing compute and training steps and by reporting results of training TTT memory under our recipe; we then explain why the RoboMME TTT baselines are not explained by insufficient training; finally, we answer Weakness 1 on how the 35-point gap is distributed.

Compute and training steps (Question 1). According to the RoboMME paper (Sec. 5.1, Appendix B.2.3):

RoboMME TTT variantsOther RoboMME variants (incl. π0.5)T²Mem
Training steps80,00080,000Stage1 20,000 + Stage2 10,000
Hardware and duration~5–6 days (mostly 4×A40)4×A40, ~3–4 days32×H200, 48 hours

The difference in hardware cost stems from the training procedure itself: each Stage2 sample in T²Mem is a complete episode (up to 1,104 time positions), and full BPTT is performed over the entire episode so that late action supervision can propagate back to early memory writes. We therefore use fewer optimization steps, but each step is more expensive; this cost is a structural property of the method rather than additional tuning or longer optimization. The real question is whether this cost explains the performance gap, and the controls below, run with the same recipe and compute, show that it does not.

Training TTT memory under our recipe (Question 1). We trained controlled TTT-memory variants under our own recipe and budget by removing T²Mem's components one at a time (same recipe, 8,000 steps), and compared them with the same-task scores of the RoboMME TTT variants in Table 1:

Removed componentTaskFull methodAfter removalRoboMME TTT variants (Table 1)
Alternating → joint trainingVideoUnmask91.33%32.67%27.22–31.78%
Observation interface → AE-side TTT integrationStopCube60.00%11.33%2.11–4.00%
Adaptive writing → fixed β=1\beta=1VideoUnmask91.33%71.33%27.22–31.78%
Adaptive writing → fixed β=1\beta=1PatternLock51.33%28.67%1.56–3.56%

The AE-side TTT integration is similar to RoboMME's TTT-Expert, placing memory on the action-expert side. The results show that, even with exactly the same recipe and compute as T²Mem, TTT memory by itself does not bring improvement: once alternating training or the observation interface is removed, success falls back to the level of the RoboMME TTT variants (e.g., 32.67% vs. 27.22–31.78% on VideoUnmask).

AuthorsOpenReview ↗

Response to Reviewer LW6g (Part 2/5)

Weakness 1 + Question 1 (continued)

Why the RoboMME TTT baselines are not explained by insufficient training. RoboMME does not report training curves, so we cannot directly verify their convergence, but three lines of evidence indicate that the issue is not training length:

  1. The RoboMME authors attribute the failure of recurrent memory to unstable training caused by fine-tuning π0.5 with shallow recurrent layers, and state that effective recurrent memory requires deeper architectural integration (Sec. 5.2). T²Mem's observation interface integrates with the backbone across 17 layers and stabilizes optimization through alternating training, directly targeting this issue.
  2. The TTT variants perform close to using no memory: their 16-task average is only about 4 percentage points above the memory-free π0.5; on VideoUnmask they score 27.22–31.78%, close to chance in a three-way choice; on StopCube they score 2.11–4.00%, even below the memory-free π0.5 (6.67%). This reflects not learning to use memory, rather than insufficient learning.
  3. Under our recipe, TTT memory without alternating training, when trained 4× longer (2,000 → 8,000 steps) on the pure memory-retrieval task VideoUnmask, only changes from 30% to 32.67%, remaining at chance level throughout.

How the 35-point gap is distributed (Weakness 1). Based on the results above, we analyze the sources of the gap as follows:

  • Training recipe and compute: The evidence indicates a small contribution. On the one hand, the memory-free Stage1 under the same recipe uses only 1/4 of the training steps of the public π0.5 (20,000 vs. 80,000) and achieves 16.38%, comparable to its 17.93%, showing that the recipe itself does not raise policy capability. On the other hand, the TTT controls trained with the same recipe and compute in the table above all fall back to the level of RoboMME TTT, showing that compute alone cannot produce the gap.
  • Observation interface and alternating training: These are the main source of the gap. Removing either one brings success back to the level of RoboMME TTT. Both the single switch (72.00%) and memory pretraining followed by joint training (75.33%) are below repeated alternation (91.33%), showing that the benefit of alternation comes from repeated alternation itself.
  • Adaptive writing: On top of this, it brings a further gain of 20–23 percentage points.

In addition, FrameSamp-Modul, the strongest non-privileged baseline in RoboMME, is also trained for 80,000 steps (44.51%), and T²Mem still outperforms it by 12.32 percentage points. We agree that the full experiment suggested by the reviewer would be the most convincing, and we carefully assessed its feasibility: rerunning the three TTT variants and adding the three components one by one would require at least six full 16-task training runs, each taking about 1,536 H200 GPU hours, which exceeds the computational resources available to us during the rebuttal period. We therefore removed components one at a time under the same recipe and used single-task controlled comparisons to answer the same question. We acknowledge that this remains a limitation of our work, and the revision will clearly distinguish external reference baselines from controlled comparisons.

Weakness 2 (opening and first point) + Question 2

We thank the reviewer for the comments.

Scale and variance. We agree that the original ablations were small in scale. The new experiments therefore cover four tasks (VideoUnmask, MoveCube, StopCube, and PatternLock), and the standard deviation across evaluation seeds is at most 6.26 percentage points for every task. In addition, single-task and 16-task performance are similar (VideoUnmask 91.33% vs. 88.00%, MoveCube 77.33% vs. 80.67%), indicating that the single-task ablations reflect multi-task behavior.

RoboTTT-style variant (Question 2). Yes. This variant includes the frozen-backbone memory training phase and uses the same alternating schedule (Appendix D): memory and policy alternate every 500 steps, the policy is frozen during memory phases, and the only difference from our method is the source of writes and queries. To further rule out insufficient training, we trained both interfaces to 8,000 steps (4× the budget). On StopCube, which requires tracking how many times an event has occurred, the AE-side variant reaches 11.33% and the observation-grounded interface reaches 60.00%, so the gap remains large. We note that this control is intended to compare interface designs under controlled conditions; the full RoboTTT method additionally includes re-pretraining on the full VLA data.

AuthorsOpenReview ↗

Response to Reviewer LW6g (Part 3/5)

Weakness 2 (second point) + Question 4

We thank the reviewer for suggesting this more informative baseline.

Single-round baseline. We added this baseline on VideoUnmask (8,000 steps, 3 seeds × 50 episodes):

Training scheduleMemory updatesPolicy updatesSuccess rate
Repeated alternation, (M500→P500)×8(M_{500} \rightarrow P_{500}) \times 84,0004,00091.33%
Single switch, M4000→P4000M_{4000} \rightarrow P_{4000}4,0004,00072.00%
Memory training with frozen policy, then joint training, M4000→J4000M_{4000} \rightarrow J_{4000}8,0004,00075.33%
Joint training from initialization——32.67%

The third row is the single-round baseline suggested by the reviewer. Repeated alternation outperforms it by 16.00 percentage points, and outperforms the single switch, which uses exactly the same number of updates for each parameter group, by 19.33 percentage points.

Number of rounds. The main experiments use ten rounds (500 memory steps and 500 policy steps per round, Appendix Table 3), and the single-task experiments use eight rounds.

Sensitivity. Since each round has a fixed length, the change with the number of rounds is the change with training steps. MoveCube reaches a high success rate within the first few rounds, while PatternLock keeps improving with the number of rounds (Stage2 steps):

Task1 round3 rounds6 rounds8 rounds
PatternLock11.33%25.33%42.67%51.33%

Weakness 2 (third point) + Question 3

We thank the reviewer for the comments. We add task success rates and explain how adaptive writing differs from existing gating rules.

Task success (Question 3). 8,000 steps, 3 seeds × 50 episodes; both settings use the same tasks, model, and evaluation protocol:

TaskAdaptive writingFixed β=1\beta=1
VideoUnmask91.33%71.33%
PatternLock51.33%28.67%

Fixed β=1\beta=1 disables the adaptive scaling and keeps the curvature calibration. These are single-task results. A full-benchmark β=1\beta=1 control requires a complete 16-task training run (about 1,536 H200 GPU hours), which could not be completed during the rebuttal period; we acknowledge this limitation, and the revision will restrict this conclusion to the single-task setting.

Not merely suppressing later writes. We agree with the reviewer that suppressing later writes almost inevitably reduces interference with early bindings, and we therefore provide the following argument. The adaptive coefficient only attenuates repeated writes aligned with the accumulated change, while keeping strong writes for new information that conflicts with stored content and has a large reconstruction residual (Sec. 3.5, Appendix A). If the gain came only from suppressing writes, PatternLock, which requires continually writing new information, should get worse; instead, it improves by 22.67 percentage points. Both tasks require writing locations into memory: VideoUnmask is an object–location binding (evidence only at the beginning), while PatternLock is a temporal–location binding (watching a pattern path drawn step by step and then reproducing it in the same order, which requires writing at different times and indexing by progress). Adaptive writing is better on both.

Differences from Titans and gated delta rules. (a) Their forget gates proportionally decay the entire memory state; our coefficient only modulates the strength of new writes and does not decay stored content, because key evidence in RoboMME often appears early and must be retained for hundreds of frames. (b) Their gates are learned functions; our coefficient is computed analytically from the update geometry and the reconstruction residual, with no additional parameters. We have not yet conducted a matched comparison with these rules; the revision will position adaptive writing as an auxiliary design, with the core contributions being the memory architecture, the observation interface, and alternating training.

Reviewer_M15QOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces T$^2$Mem, a memory-augmented VLA framework that encodes observation history into fast weights through self-supervised test-time updates. The method uses an observation-grounded interface to extract visual-language information and provide memory-augmented representations to the action expert of pi0.5. Its main training contribution is alternating memory-policy learning, which separately optimizes memory formation and memory utilization using action supervision. The authors also introduce adaptive memory writing to reduce interference between successive updates. On 16 RoboMME tasks, T$^2$Mem achieves 56.83% average success, compared with 17.93% for the published memory-free pi0.5 baseline and 44.51% for the strongest reported non-oracle baseline. Additional experiments investigate architectural choices, training strategies, causal memory dependence, and inference efficiency.

Critical strengths
  1. T$^2$Mem substantially improves success across the RoboMME benchmark, outperforming previously reported recurrent, perceptual, and non-oracle symbolic memory methods. The improvements span several memory categories, including counting, object permanence, and imitation, suggesting that the method is not limited to a single type of historical information

  2. The counterfactual memory experiments are particularly valuable. On VideoUnmask, replacing correct memory with conflicting memory reduces success from 300/300 to 1/300, while disabling online writes causes substantial performance degradation

Critical weaknesses
  1. The training sequence construction and long-horizon credit assignment remain insufficiently specified. Although Appendix A clarifies that gradients propagate through fast-weight updates, and Appendix B provides the overall training schedule, the actual construction of Stage 2 training sequences remains unclear. Are complete trajectories processed sequentially in time (for each episode t0 -> t1 -> t2 -> ... without shuffling timesteps), or are fixed-length windows sampled like in RoboMME? Which observations are included, how frequently are memory writes performed during training, and at which frames is the action loss applied? The normalization of losses across variable-length sequences is also unclear. These details directly determine whether future action supervision can influence memory writes made hundreds of steps earlier. Without an explicit training algorithm and sequence-sampling protocol, faithfully reproducing the central learning mechanism remains difficult.

  2. Several architectural and optimization details remain ambiguous despite the extensive appendix. The appendix specifies memory dimensions, parameter counts, write equations, freezing schedules, and learning rates, which is useful. However, the precise ordering of attention residuals, memory fusion, and feed-forward transformations within the interface is not fully defined. It is also unclear whether the fast-weight MLP includes a residual connection, how the additional proprioceptive projection interacts with the original pi0.5 state representation, and which modules receive LoRA adapters. Important optimizer details, including weight decay, gradient clipping, learning-rate scheduling, and optimizer-state handling across alternating phases, are also missing. Given the complexity of the architecture and its approximately 68M additional parameters, these choices can materially affect reproducibility.

  3. The methodological contributions are not sufficiently isolated or compared with prior TTT-based approaches. RoboTTT and WAM-TTT (Feng et al., 2026) already establishes fast-weight memory with self-supervised updates and action-supervised sequence training for robot policies. The main distinguishing components of T$^2$Mem therefore appear to be its observation-grounded interface, alternating optimization, and adaptive writing. The authors provide a controlled comparison with an AE-side RoboTTT-style implementation and with joint optimization, which is valuable. However, both comparisons are limited to two tasks with task-specific checkpoints. In particular, it remains unclear whether repeated alternation is necessary or whether sequential memory-then-policy training or better-tuned joint optimization could achieve similar results. Furthermore, adaptive writing is evaluated through reconstruction NMSE rather than downstream task success. More comprehensive ablations are needed to establish the independent contributions of the proposed mechanisms.

  4. The main baseline comparisons are not fully controlled. All baseline scores in Table 1 are taken from the original RoboMME paper rather than reproduced under the same training setup. The authors own memory-free Stage 1 achieves 16.38% average success, compared with 17.93% for the published pi0.5 baseline. While the improvement of T$^2$Mem remains substantial, differences in initialization, training budgets, action execution, and optimization may affect the comparison. A matched evaluation against strong recurrent and TTT-based policies would provide stronger evidence of the proposed method's advantage. The paper also does not report sufficient variability across independent training runs.

  5. Memory retention is demonstrated more convincingly than memory updating and long-horizon scalability. T$^2$Mem performs substantially worse on tasks such as VideoUnmaskSwap and ButtonUnmaskSwap, which require updating previously established object-location associations. The authors attribute these failures partly to limitations of the underlying policy's temporal reasoning, but do not distinguish incorrect memory updates from failures to use correctly updated information. This is particularly relevant because adaptive writing explicitly reduces repeated updates and could potentially interfere with overwriting outdated information. Moreover, the learned fast state contains approximately 9.5M FP32 parameters per episode, and training requires 32 H200 GPUs for 48 hours. The paper does not systematically evaluate memory capacity, performance over longer horizons, or training and inference costs as the number of historical events increases.

  6. The experimental evaluation is restricted to RoboMME, leaving the generality of the proposed memory mechanism insufficiently established. MIKASA-Robo (Cherepanov et al., 2026) provides a complementary benchmark with systematically designed memory-intensive manipulation tasks, while its VLA extension covers a broader range of memory types and temporal horizons. Evaluating T$^2$Mem on these tasks and comparing it against relevant memory-augmented baselines under matched training and execution protocols would provide stronger evidence of generalization. This is particularly important for assessing long-horizon retention, memory updating, and the ability to handle memory structures not represented in RoboMME. Moreover, it would be great to see that the use of TTT does not lead to performance degradation on Markovian benchmarks such as LIBERO (Liu et al., 2023).

Questions for authors
  1. Training sequences and BPTT. What exactly constitutes a Stage 2 training sequence: a complete episode or a sampled window? Are observations processed at every timestep or using the inference write stride of 16? At which frames is the action loss computed, and how is it averaged across frames and variable-length episodes? Is flow time and Gaussian noise sampled independently for each supervised action chunk? What is the maximum gradient propagation horizon?

  2. Memory interface architecture. Could you specify the exact ordering of attention, residual connections, memory retrieval, gated fusion, and feed-forward transformations within each interface layer? Is the fast-weight MLP implemented with or without a residual connection?

  3. pi0.5 integration. How is proprioception represented during Stage 1 versus Stage 2? Does the additional continuous state projection replace or coexist with the original pi0.5 state tokens? Could you also specify the exact pretrained checkpoint, VLM LoRA configuration (rank, alpha, target modules), and whether the action expert is fully fine-tuned?

  4. Optimization details. What optimizer, weight decay, gradient clipping, learning-rate scheduling, and EMA settings are used in each stage? Are optimizer states preserved or reset between successive memory and policy phases? For the per-head adaptive writing computations, are the weights and biases treated as one concatenated parameter vector or handled separately?

  5. Alternating learning. How does repeated alternating optimization compare with a single memory-training phase followed by policy fine-tuning? Could you also evaluate joint optimization with separately tuned learning rates or other stabilization strategies? This would help determine whether repeated alternation itself is responsible for the large improvements.

  6. Comparison with prior methods. Beyond the observation-grounded interface and training schedule, what are the main methodological differences from RoboTTT? Could you provide matched comparisons with the AE-side TTT integration and other strong recurrent-memory baselines across the full RoboMME benchmark, rather than only two tasks?

7 . *Baseline comparability. How closely are the published RoboMME baselines matched in terms of pretrained checkpoints, training data, optimization budgets, action horizons, and execution protocols? Why does your Stage 1 memory-free policy differ from the published pi0.5 baseline? Could you report results over independent training seeds?

  1. Memory updating and adaptive writing. Can you distinguish memory-update failures from downstream reasoning failures on the swap tasks? Does adaptive writing improve task success compared with fixed write scaling, and how does it affect the ability to overwrite outdated associations? Controlled experiments with memory probes before and after object swaps would be particularly informative.

  2. Scalability and computational efficiency. How do memory retention, task success, training GPU memory, and inference latency scale with the number of historical events and sequence length? Given the 9.5M-parameter fast state and the reported training cost, what are the practical limits of the method compared with simpler recurrent or keyframe-based memory mechanisms?

10 comments
AuthorsOpenReview ↗

Response to Reviewer M15Q (Part 1/6)

We sincerely thank the reviewer for the detailed and constructive review. We are glad that the reviewer recognizes that T²Mem substantially improves success across the RoboMME benchmark, outperforming previously reported recurrent, perceptual, and non-oracle symbolic memory methods, with improvements spanning several memory categories including counting, object permanence, and imitation. We also appreciate that the reviewer finds the counterfactual memory experiments particularly valuable, where replacing correct memory with conflicting memory reduces success from 300/300 to 1/300 and disabling online writes causes substantial degradation. Below we respond to each weakness and question in turn.

Weakness 1 + Question 1

We thank the reviewer for the attention to reproducibility. The overall training pipeline is shown in Appendix B, Fig. 7. We provide the requested details below, which will also be added to the appendix of the revision:

  • Sequences: Each Stage2 sample is a complete, temporally ordered episode; we neither crop windows nor shuffle time steps.
  • Write interval: Demonstration observations are selected on the write grid, followed by per-frame execution observations; the write interval is 16, identical to inference.
  • Action loss: The action loss is computed on all valid execution frames and not on demonstration frames. The loss is averaged first within each chunk (over valid elements), then within each episode (by supervision weight), and finally over the batch; padding is neither written nor included in the loss, so long episodes do not receive higher weight.
  • Noise: Flow time and Gaussian noise are sampled independently for each action chunk.
  • Gradient range: We do not use TBPTT; gradients flow through the entire episode (rematerialization only saves memory and does not truncate), up to 1,104 time positions. The adaptive coefficient and the curvature scalar use stop-gradient, while the fast-weight recurrence remains differentiable.

Hence, the long-horizon credit assignment the reviewer asks about holds: action supervision on late execution frames can propagate through the full fast-weight recurrence back to memory writes hundreds of steps earlier.

Weakness 2 + Question 2 + Question 3 + Question 4

We thank the reviewer for the detailed questions. We agree that these details are important for reproducibility, and all of the following will be added to the appendix of the revision.

Interface architecture (Question 2). The order within each interface layer is: pre-norm → attention → memory read/write using the interface attention output (read before write) → gated fusion a+tanh⁡(α)⊙ra+\tanh(\alpha)\odot r → attention residual → pre-norm → FFN → FFN residual. Interface tokens attend only to vision-language features; action and proprioception tokens are not used as TTT keys, values, or queries. The fast-weight MLP has no input residual: fW(x)=W2 GeLU(W1x+b1)+b2f_W(x)=W_2\,\mathrm{GeLU}(W_1x+b_1)+b_2.

π0.5 integration (Question 3).

  • Proprioception: Stage1 has no additional state token. In Stage2, the normalized continuous state is projected into one token, which forms the suffix together with the 16 interface tokens and the action tokens; it does not coexist with the discrete state prompt.
  • Initialization: The official π0.5 base checkpoint (gs://openpi-assets/checkpoints/pi05_base). Stage2 starts from Stage1 at step 20,000, with newly initialized memory parameters.
  • LoRA: The VLM uses gemma_2b_lora (attention and FFN, rank/alpha 128); the interface attention adapter uses rank/alpha 16 (Appendix Table 2). The base VLM and the vision encoder are frozen.
  • Action expert: Fully fine-tuned in policy phases and frozen in memory phases; gradients still flow through it to the memory.

Optimization (Question 4).

  • AdamW (first/second moment coefficients 0.9/0.95, epsilon 10−810^{-8}, weight decay 10−1010^{-10}), gradient clipping at 1.0; EMA is disabled in the new experiments.
  • Learning rates: listed in Appendix Table 3, with 25 warmup steps in each 500-step phase.
  • Memory and policy each maintain persistent Adam states, which are not reset when switching phases.
  • The per-head adaptive coefficient is computed jointly over W1,b1,W2,b2W_1, b_1, W_2, b_2, which is equivalent to concatenating them into a single vector.
AuthorsOpenReview ↗

Response to Reviewer M15Q (Part 2/6)

Weakness 3 + Question 5 + Question 6

We thank the reviewer for the comments. Below we address, in order, the differences from prior TTT methods, the necessity of repeated alternation, the interface comparison and task results for adaptive writing, and the full-benchmark comparison.

Differences from prior methods. We first clarify that fast-weight memory with self-supervised online updates comes from prior TTT work and is not our contribution; our contribution is how memory formation and use are organized inside a VLA, namely the observation-grounded interface and alternating training. Specifically:

  • WAM-TTT uses unlabeled human videos at test time to steer a world-action model toward new task variants. This is cross-episode skill adaptation and requires meta-training on paired human–robot data. In contrast, TTT in T²Mem is within-episode memory, reset to W0W_0 at the start of every episode, and requires no additional data.

  • RoboTTT shares fast-weight memory and the gate initialization with T²Mem (we follow RoboTTT for the latter), but the two differ fundamentally in three respects:

    • Different objectives. RoboTTT targets visuomotor context scaling and in-context adaptation, so that action prediction can exploit a longer history. T²Mem targets memory-dependent decision making: key cues often appear only once (e.g., the color–container correspondence before occlusion, or how many times an event has occurred), after which the current observation no longer contains them, and the policy must retrieve them accurately. We believe that a longer context does not equal accurate memory indexing.
    • Different memory location and content. RoboTTT introduces TTT into the action expert, and memory reads and writes are both derived from action-expert representations, so the memory mainly stores action- and motion-level history. T²Mem places the memory interface on the VLM side, where 16 learned query tokens extract task-relevant semantics from vision-language features before writing, and the readout is then consumed by the action expert. Our memory therefore stores what was seen, not only what was done.
    • Different training cost. RoboTTT requires re-pretraining on the full VLA pretraining data. T²Mem is added directly to an existing π0.5 and uses alternating training to adapt memory and policy to each other, requiring only fine-tuning on a small set of memory tasks.

    The interface comparison below changes only the source of memory reads and writes while keeping all other settings identical, and is designed precisely to test the second difference.

AuthorsOpenReview ↗

Response to Reviewer M15Q (Part 3/6)

Weakness 3 + Question 5 + Question 6 (continued)

Is repeated alternation necessary? (Question 5) To answer this question, we compared four training schedules on VideoUnmask (8,000 steps, 3 seeds × 50 episodes):

Training scheduleMemory updatesPolicy updatesSuccess rate
Repeated alternation, (M500→P500)×8(M_{500} \rightarrow P_{500}) \times 84,0004,00091.33%
Single switch, M4000→P4000M_{4000} \rightarrow P_{4000}4,0004,00072.00%
Memory pretraining followed by joint optimization, M4000→J4000M_{4000} \rightarrow J_{4000}8,0004,00075.33%
Joint training from initialization——32.67%

Memory pretraining followed by joint optimization uses group-wise learning rates and preserves optimizer states, corresponding to the stabilized joint training suggested by the reviewer. More importantly, the single switch and repeated alternation use exactly the same number of updates for each parameter group and differ only in the number of alternations, yet the latter is 19.33 percentage points higher. This shows that repeated alternation itself is a major source of the improvement.

Interface comparison (Question 6). Besides the training schedule, the other main difference from RoboTTT is where the memory interface is placed. To isolate this factor, the AE-side variant uses the same training framework as ours and changes only the source of writes and queries. Results are shown below (StopCube requires tracking how many times an event has occurred; 8,000 steps, 3 seeds × 50 episodes):

TaskObservation-grounded interfaceAE-side
StopCube60.00%11.33%

Adaptive writing (Weakness 3). Since adaptive writing was evaluated only by reconstruction NMSE, we added task success rates (8,000 steps, 3 seeds × 50 episodes):

TaskAdaptive writingFixed β=1\beta=1
VideoUnmask91.33%71.33%
PatternLock51.33%28.67%

Full benchmark. On the full RoboMME benchmark, T²Mem outperforms the best TTT variant in RoboMME (22.35%) by about 34.5 percentage points, with about 3.5× faster inference. We recognize the value of a matched full-benchmark comparison and would very much like to add it. However, each 16-task training run takes about 1,536 H200 GPU hours, and matched comparisons against AE-side TTT integration and other recurrent-memory baselines would require several such full runs, which exceeds the computational resources available to us during the rebuttal period. We therefore conducted rigorous controlled comparisons of each component in the single-task setting. We acknowledge this limitation, and the revision will restrict component-level conclusions to the tasks actually tested.

Reviewer_FoL2OpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

T²MEM adds fast-weight memory to a pretrained vision-language-action policy for tasks that require information no longer available in the current observation. Its main architectural contribution is an observation-grounded interface that extracts visual-language features, writes them into memory, and provides retrieved context to the action expert. The method alternates between training the memory mechanism and adapting the policy to use it. At deployment, memory updates use observations alone, without memory-specific labels or an external reasoning model.

The paper evaluates the method on 16 RoboMME tasks and includes ablations of the memory architecture and training schedule, as well as interventions on memory writes, content, and timing.

Critical strengths
  • The motivation is well grounded, and the paper tests whether the policy actually uses memory.

  • The proposed architecture and training schedule have direct experimental support.

Critical weaknesses
  1. The evidence for the individual design choices is narrower than the main evaluation. Both architectural and training ablations use separately trained models on only VideoUnmask and MoveCube, while the headline results use a 16-task model. The controlled comparisons are valuable, but it remains unclear whether the same conclusions hold in the multi-task setting. The proposed explanations also need some qualification. The AE-side comparison supports the new interface as a whole; it does not isolate semantic feature quality from the other changes in information routing. Likewise, the joint-training result establishes a performance gap under the tested recipe, but does not establish that changing memory–policy representations is the cause of that gap. Evidence that the joint baseline was reasonably tuned and had converged would make this argument stronger.

  2. The adaptive-writing experiment supports retention, but not yet the benefit of adaptation itself. Figure 4(c) shows that adaptive writing preserves early K/V bindings better than setting (\beta_t=1). However, the adaptive coefficient always lies between 0.1 and 1, so it also reduces write strength. A smaller fixed coefficient might preserve old bindings better for the same reason. The current experiment does not distinguish this possibility from a benefit of the proposed adaptive rule. It also does not report downstream success rates, including on tasks where old bindings must be updated. This is a secondary limitation: it weakens the evidence for this component, but does not undermine the main architectural result.

Questions for authors
  1. Do the architectural and alternating-training gains in Figure 4 persist in the multi-task setting used for Table 1? A matched AE-side or joint-training comparison in that setting would help connect the component ablations to the main result.

  2. Does adaptive writing improve task success relative to both (\beta_t=1) and a smaller fixed write coefficient? In particular, does it help when the policy must update an existing object–location binding, rather than retain an unchanged early cue?

4 comments
AuthorsOpenReview ↗

Response to Reviewer FoL2 (Part 1/3)

We sincerely thank the reviewer for the careful and constructive review. We are glad that the reviewer finds the motivation well grounded and appreciates that the paper tests whether the policy actually uses memory, and that the reviewer considers the proposed architecture and training schedule to have direct experimental support. Below we respond to each weakness and question in turn.

Weakness 1 + Question 1

Weakness 1: The evidence for the individual design choices is narrower than the main evaluation. Both architectural and training ablations use separately trained models on only VideoUnmask and MoveCube, while the headline results use a 16-task model. The controlled comparisons are valuable, but it remains unclear whether the same conclusions hold in the multi-task setting. The proposed explanations also need some qualification. The AE-side comparison supports the new interface as a whole; it does not isolate semantic feature quality from the other changes in information routing. Likewise, the joint-training result establishes a performance gap under the tested recipe, but does not establish that changing memory–policy representations is the cause of that gap. Evidence that the joint baseline was reasonably tuned and had converged would make this argument stronger.

Question 1: Do the architectural and alternating-training gains in Figure 4 persist in the multi-task setting used for Table 1? A matched AE-side or joint-training comparison in that setting would help connect the component ablations to the main result.

We thank the reviewer for recognizing the value of the controlled comparisons and for the detailed comments on the scope and attribution of our conclusions. We respond in four parts.

(1) Multi-task setting. We understand the reviewer's concern that the ablations use single-task models while the main results come from the 16-task model. To address this, we retrained single-task models on the two ablation tasks so that their training length matches the new controlled experiments. The single-task and 16-task models perform similarly: 91.33% vs. 88.00% on VideoUnmask and 77.33% vs. 80.67% on MoveCube. The 16-task model also shows clear gains over its memory-free Stage1 (27.33% and 38.00%, respectively). This indicates that the single-task setting reflects multi-task behavior well. We would very much like to run matched ablations in the 16-task setting, but each 16-task training run takes 32 H200 GPUs for 48 hours (about 1,536 GPU hours), and every ablation requires a separate full run, which exceeds the computational resources available to us during the rebuttal period. We therefore chose to conduct controlled comparisons that are as rigorous as possible in the single-task setting. We acknowledge that this is a genuine limitation of our work, and in the revision we will restrict the component-level conclusions to the settings actually evaluated.

AuthorsOpenReview ↗

Response to Reviewer FoL2 (Part 2/3)

Weakness 1 + Question 1 (continued)

(2) AE-side comparison. We agree that what this comparison tests should be stated more clearly. In both settings, the memory readout is consumed by the action expert (AE) (Fig. 2, Sec. 3.3); the only difference is the source of writes and queries: our interface takes them from the VLM's vision-language features, whereas the AE-side variant takes them from AE tokens. The comparison therefore tests whether memory is extracted from vision-language features. To strengthen the evidence, we added an experiment on a different type of memory task: on StopCube, which requires tracking how many times an event has occurred (8,000 steps), the observation-grounded interface reaches 60.00%, while the strictly controlled AE-side variant reaches only 11.33%.

(3) Convergence of the joint baseline. To address whether the joint baseline was adequately trained, we extended both schemes from 2,000 steps in the original paper to 8,000 steps (3 seeds × 50 episodes). Both improve: repeated alternation goes from 84% to 91.33% on VideoUnmask and from 71% to 77.33% on MoveCube, and joint training on MoveCube clearly exceeds the memory-free baseline (38.00% for the 16-task Stage1). Nevertheless, repeated alternation remains higher on both tasks, and its advantage does not change with training length:

TaskTraining stepsRepeated alternationJoint training
VideoUnmask8,00091.33%32.67%
MoveCube8,00077.33%66.67%

We further compared two stronger alternatives (VideoUnmask, 8,000 steps):

Training scheduleSuccess rate
Joint training from initialization32.67%
Single switch, M4000→P4000M_{4000} \rightarrow P_{4000}72.00%
Memory pretraining followed by joint optimization, M4000→J4000M_{4000} \rightarrow J_{4000}75.33%
Repeated alternation, (M500→P500)×8(M_{500} \rightarrow P_{500}) \times 891.33%

Memory pretraining substantially improves joint training, but repeated alternation remains the best. The revision will report these 8,000-step results.

(4) Cause of the gap. We agree with the reviewer that the current experiments establish the performance gap but are not sufficient to establish its causal mechanism. In the revision, we will present the co-adaptation of memory and policy representations as an explanation of the gap rather than a verified causal conclusion.

AuthorsOpenReview ↗

Response to Reviewer FoL2 (Part 3/3)

Weakness 2 + Question 2

Weakness 2: The adaptive-writing experiment supports retention, but not yet the benefit of adaptation itself. Figure 4(c) shows that adaptive writing preserves early K/V bindings better than setting βt=1\beta_t=1. However, the adaptive coefficient always lies between 0.1 and 1, so it also reduces write strength. A smaller fixed coefficient might preserve old bindings better for the same reason. The current experiment does not distinguish this possibility from a benefit of the proposed adaptive rule. It also does not report downstream success rates, including on tasks where old bindings must be updated. This is a secondary limitation: it weakens the evidence for this component, but does not undermine the main architectural result.

Question 2: Does adaptive writing improve task success relative to both βt=1\beta_t=1 and a smaller fixed write coefficient? In particular, does it help when the policy must update an existing object–location binding, rather than retain an unchanged early cue?

We thank the reviewer for raising this point. To directly answer whether adaptive writing improves task success, we added task evaluations with a fixed β=1\beta=1 (8,000 steps, 3 seeds × 50 episodes). Fixed β=1\beta=1 removes only the adaptive scaling and keeps the curvature-calibrated step size, so the difference between the two settings comes solely from whether the write coefficient is adaptive:

TaskAdaptive writingFixed β=1\beta=1
VideoUnmask91.33%71.33%
PatternLock51.33%28.67%

Two types of location binding. These two tasks correspond to different memory demands. Both VideoUnmask and PatternLock require writing location bindings into memory; they differ in the key of the binding. VideoUnmask is an object–location binding: before occlusion, the policy must remember which color is under which container; the evidence appears only once at the beginning and afterwards only needs to be retained. PatternLock is a temporal–location binding: the robot watches a pattern path drawn step by step in a demonstration and then reproduces it in the same order. Each location on the path must be written with its order of appearance as the key; new bindings arriving at different times must be written sufficiently without overwriting earlier ones; and during reproduction, the next location must be retrieved according to the current progress. Adaptive writing improves success by 20.00 and 22.67 percentage points on the two tasks, respectively, showing that its benefit is not limited to retaining an unchanged early cue but also covers continually writing new location bindings and indexing them by progress.

Is the gain only due to weaker writes? The reviewer is concerned that the gain may simply come from reduced write strength. In fact, the adaptive coefficient does not uniformly shrink writes: it attenuates repeated writes aligned with the accumulated change Wt−W0W_t-W_0, while keeping strong writes for new information that is inconsistent with stored content and has a large reconstruction residual (Sec. 3.5, Appendix A). If the gain came only from weaker writes, success should drop on tasks that require continually writing new evidence; instead, on PatternLock, success increases from 28.67% to 51.33%.

When bindings must be updated. Regarding the reviewer's specific concern about binding updates: in RoboMME, the tasks that require updating existing object–location bindings include VideoUnmaskSwap and ButtonUnmaskSwap, where objects swap positions after being memorized. The adaptive rule does not suppress such updates: post-swap observations conflict with stored associations and produce large reconstruction residuals, so the write coefficient stays high; only repeated writes aligned with existing changes are attenuated. On these two tasks, T²Mem still clearly improves over the memory-free Stage1 (29.33% vs. 18.00% and 20.00% vs. 2.00%). The low absolute success on swap tasks is a difficulty shared by all methods: every non-privileged method stays at or below 38%, including FrameSamp, which can access past frames, and MemER, which uses an external VLM; only GroundSG Oracle, which is given ground-truth locations, solves them (99.22% and 80.22%). This indicates that the main bottleneck lies in inferring the target's current location from the swap, not in suppressed writing.

We will report these task success rates in the main text and restrict the conclusion of Fig. 4(c) (18.8–46.9% lower early-binding error) to improved retention under replay interventions.

We thank the reviewer again for the time and the constructive comments. In our responses, we have added the requested experiments and clarified the points raised. We hope these have addressed all concerns. If anything remains unclear, we would be very happy to provide further clarification.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.