acceptodds
Under review as a conference paper at ICLR 2027

Test-Time Training as Residual Memory for Robot Policies

Abstract

Memory is essential for long-horizon robotic manipulation, where successful actions may depend on past events that are no longer recoverable from the current observation. As episodes grow longer, however, retaining the full history becomes increasingly costly, creating a fundamental scalability challenge for memory-augmented policies. Existing approaches address this challenge by either storing selected past observations in a bounded memory bank or compressing interaction history into a fixed-size parametric state through Test-Time Training (TTT). Yet these formulations do not explicitly distinguish between historical information that can already be recovered from the policy's current context and information that must persist beyond it. We introduce TTT-RM, which repurposes TTT as Residual Memory, using fast weights not to generically compress history but to complement a bounded memory bank by preserving task-relevant historical information that cannot be recovered from the policy's current context. Concretely, TTT-RM learns a history decoder that reconstructs historical representations from the current observation and retrieved memory. The resulting reconstruction residual captures what this context fails to explain and serves as the learning target for TTT. The TTT slow weights are optimized against this residual target so that online fast-weight updates learn to encode complementary historical information over time. The fast-weight state is then queried to produce a residual memory representation that conditions action generation. Extensive experiments on memory-intensive simulation benchmarks and real-world tasks show that TTT-RM consistently improves across multiple memory designs, outperforms diverse baselines, and supports sustained execution on a three-minute, eight-stage stowing task.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Robot learning, Test-Time Training

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_P1pbOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes TTT-RM, a framework that combines Explicit Memory (EM) with Test-Time Training to retain residual historical information that is not recoverable from the current observation and retrieved memory. A history decoder reconstructs past visual representations from this context, and its reconstruction residual supervises the change in TTT readouts relative to the episode's initial state. Incoming visual features update the fast weights during each episode, while action and residual objectives jointly train the slow parameters governing memory writing and retrieval. The method is evaluated on RoboMME, RMBench, and three real-world manipulation tasks, with additional ablations and latency analysis.

Critical strengths
  • The paper provides a precise and clearly presented formulation of memory-conditioned action policies, Explicit Memory (EM), and Test-Time Training (TTT). Especially, the distinction between online fast-weight updates and outer-loop learning, together with the staged training procedure, makes the proposed framework easy to follow.
  • The proposed method is comprehensively evaluated across two simulation benchmarks and real-world manipulation tasks with different memory requirements, including counting and tracking task progress. The reported improvements over EM baselines across these settings demonstrate the practical potential of combining explicit and parametric memory for history-dependent control.
Critical weaknesses
  • The paper does not sufficiently establish whether failures stem from missing historical information, inadequate use of retained context, or difficulties mapping that context to low-level actions. Controlled analyses of these alternatives would better justify the need for additional historical feature reconstruction.
  • The EM and TTT baseline configurations differ across RoboMME, RMBench, and real-world tasks, and some baseline details are not specified (e.g., the TTT variant used on RMBench and the real-world EM's selection rules and memory budget). These omissions limit reproducibility and make it difficult to assess performance across domains under a consistent memory setup.
Questions for authors
  • Could the authors provide rollout examples identifying past timesteps and visual patches with large decoder reconstruction errors, and show how TTT readouts recover task-critical information unavailable from the current observation and EM? Furthermore, how do these examples distinguish missing historical information from decoder approximation error or task-irrelevant visual variation?
  • In Figure 3(b), how would TTT-RM compare with an EM+TTT baseline trained to reconstruct full historical visual representations under the same setup? Including this baseline would clarify the specific benefit of residual targets beyond historical reconstruction supervision.
6 comments
AuthorsOpenReview ↗

Rebuttal by Authors (1)

Thank you for your insightful review! In response, we have conducted additional controlled ablations and analyses, clarified our experimental setups and implementation details to improve reproducibility, and revised the manuscript accordingly. We hope these additions address your concerns and welcome further discussion.

W1. Justifying the need for additional historical reconstruction

Thank you for this insightful suggestion. To investigate whether missing historical information contributes to policy errors, we conducted additional analyses examining how historical information availability affects reconstruction, action prediction, and downstream performance.

First, we examine how incomplete historical coverage affects reconstruction. At an EM budget of K=512K=512, we evaluate the frozen history decoder on 256 episode contexts held out from decoder training. We compare normalized reconstruction errors between historical patches whose source features are fully retained in Explicit Memory (EM) and those absent from EM. Reconstruction error is 1.75× higher for FrameSamp and 1.94× higher for TokenDrop on absent patches. These findings indicate that limited historical coverage leaves information that is substantially more difficult to reconstruct from the existing context.

Second, we directly assess the contribution of historical information to action prediction. We conduct controlled historical-cue withholding experiments under FrameSamp, keeping the current observation and policy parameters fixed while removing historical cues. This increases expert-action prediction loss by 6.03% for EM and 5.80% for TTT-RM. These results demonstrate that access to historical information contributes to action prediction independently of changes to policy parameters, rather than being solely a matter of learning to map a fixed context into actions.

Third, we examine whether the benefit of RM changes with historical information availability. We evaluate FrameSamp–Modul on RoboMME with memory budgets K∈K\in{256,512,1024}, keeping historical selection rules, training budgets, and evaluation episodes matched across methods. TTT-RM outperforms EM at every evaluated capacity, with its numerical advantage decreasing from 2.8 to 1.5 percentage points as KK increases. This trend supports our motivation for complementary historical memory: RM provides greater additional benefit when EM retains less history, while its marginal contribution diminishes as more historical information becomes available through EM.

Table R1. Effect of EM capacity on RoboMME (success rate, %).

Memory budget KKEMTTT-RMImprovement
25634.7 ± 0.737.5 ± 0.6+2.8 pp
51242.8 ± 1.945.4 ± 1.0+2.6 pp
102444.4 ± 1.445.9 ± 0.8+1.5 pp

Together, these findings identify limited historical access as a meaningful contributor to history-dependent control errors. Historical information absent from EM is harder to reconstruct, removing historical cues degrades action prediction under a fixed policy, and RM's marginal benefit decreases as EM capacity increases. These results motivate complementary historical reconstruction without assuming that missing history accounts for every failure. We further examine what RM readouts recover and their action relevance in our response to Q1.

AuthorsOpenReview ↗

Rebuttal by Authors (2)

W2. Baseline configurations and reproducibility

We use benchmark-specific Explicit Memory configurations to evaluate TTT-RM across different memory architectures, while ensuring that TTT-RM and its corresponding EM baseline use identical memory representations, capacities, selection rules, and retrieval mechanisms within each benchmark. This allows us to isolate the contribution of Residual Memory (RM) without confounding it with changes to the underlying EM design. We clarify the configurations below.

1. Simulation benchmarks. On RoboMME, we evaluate TokenDrop and FrameSamp under both Expert and Modul integration architectures, all with the same visual-token budget of K=512K=512. The standalone TTT baselines follow RoboMME's original Expert and Modul configurations. On RMBench, our main EM baseline uses FrameSamp-style memory with Modul integration and K=512K=512. We additionally evaluated TokenDrop w/ Modul on RMBench, where TTT-RM improves over its EM baseline from 31.2% to 36.4% on M(1) and from 9.7% to 12.0% on M(n). For the standalone TTT baseline on RMBench, we reimplement RoboTTT with a separate fast-weight MLP in each of the 18 action-expert layers, a learned write-rate gate, and gated residual integration of memory readouts.

2. Real-world experiments. EM maintains an episodic bank of up to 72 interaction records, each encoded as a 256-dimensional token representing the observation, commanded action, and resulting outcome. One record is created per action chunk and finalized when the subsequent observation becomes available. Records are retained using a first-in, first-out (FIFO) policy when the bank reaches capacity. The policy accesses all retained records through learned cross-attention, without additional keyframe selection or top-kk filtering. TTT-RM uses this same EM configuration and memory budget as the real-world EM baseline. The standalone TTT baseline follows our RoboTTT reimplementation described above.

Thus, although the memory representations differ across benchmarks, each EM–TTT-RM comparison holds the underlying explicit-memory configuration fixed, allowing performance differences to be attributed to the addition of RM under the respective training protocols.

AuthorsOpenReview ↗

Rebuttal by Authors (3)

Q1. Historical reconstruction and Residual Memory readouts

Thank you for this valuable suggestion. We conducted additional quantitative analyses and rollout visualizations to examine (1) what historical information Residual Memory (RM) recovers, (2) how these improvements relate to missing Explicit Memory (EM) coverage, and (3) whether the learned readouts provide information useful for control.

1. Historical feature recovery and rollout visualization. We compare decoder-only reconstruction y^\hat{\mathbf{y}} with reconstruction augmented by RM readouts y^+δ\hat{\mathbf{y}}+\boldsymbol{\delta}. Across 16,352 historical-address evaluations from 256 decisions spanning 16 RoboMME tasks, RM reduces historical-feature reconstruction MSE by 2.80% under TokenDrop. This provides direct evidence that RM readouts recover information unexplained by the decoder, supporting their role as complementary memory rather than merely an additional action-conditioning pathway.

We additionally introduce Figure 6 (revised Appendix D), visualizing patch-level reconstruction improvements across representative RoboMME rollouts. In ButtonUnmaskSwap, improvements appear around button-pressing and container-unmasking regions, with spatial patterns changing across task stages. In PickXtimes, improvements appear around cube manipulation during repeated placements and the stopping button at task completion. These task-associated patterns provide qualitative evidence that RM recovers historical features relevant to task progression.

2. Distinguishing missing history from decoder approximation error. As detailed in W1, we examine reconstruction errors according to EM coverage. At K=512K=512, across 256 held-out contexts, decoder reconstruction errors are 1.75× higher for FrameSamp and 1.94× higher for TokenDrop when historical patches are absent from EM than when fully retained. This association indicates that limited historical coverage contributes to reconstruction difficulty.

3. Action relevance of recovered information. To assess whether RM readouts provide action-relevant information beyond visual reconstruction, we conduct held-out action-prediction probing, comparing a fixed context representation alone, augmented with correct RM readouts, or augmented with shuffled readouts from episodes sharing the same task and instruction.

Table R2. Action-prediction probing on held-out episodes (lower MSE is better).

ComparisonAction MSEReduction with correct RM
Context only → Context + RM0.09215 → 0.087774.76%
Context + shuffled RM → Context + correct RM0.10033 → 0.0877712.52%

Correct and shuffled RM readouts use the same trained probe and normalization. The improvement over context alone demonstrates that RM provides additional action-predictive information, while the improvement over instruction-matched shuffled readouts indicates that this information is specific to the corresponding episode rather than merely reflecting the task or additional feature dimensions.

Together, these analyses support RM's role in recovering complementary historical information useful for control. RM readouts recover features unexplained by the decoder, improve reconstruction in task-associated regions, and provide additional action-predictive information.

Q2. Comparison with full-history reconstruction

Thank you for this suggestion. To isolate the benefit of residual supervision, we conducted the full-history reconstruction ablation (r=y\mathbf{r}=\mathbf{y}), replacing the decoder-defined residual target (r=y−y^\mathbf{r}=\mathbf{y}-\hat{\mathbf{y}}) while keeping the architecture, action objective, auxiliary-loss weight, training and evaluation protocols, and other hyperparameters unchanged.

Table R3. Residual versus absolute-history reconstruction (success rate, %).

MethodEM + TTTAbsolute (r=yr=y)Residual (r=y−y^r=y-\hat y)
FrameSamp–Modul42.840.646.6
TokenDrop–Modul33.132.838.4

Residual supervision outperforms full-history reconstruction by 6.0 and 5.6 percentage points, respectively, while absolute reconstruction slightly underperforms naive EM+TTT in both configurations. A possible explanation is that absolute reconstruction allocates limited fast-weight capacity to information already recoverable from the current observation and EM, introducing redundant supervision that may interfere with action optimization. In contrast, the decoder-defined residual target emphasizes information unexplained by the existing context, encouraging complementary memory representations. This validates our central design choice of using context-conditioned residual supervision to encourage complementary memory representations. We have incorporated this comparison into the revised ablation section.

Reviewer_LP5NOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper adds a TTT fast-weight memory to a policy that already has an explicit memory bank (EM). A history decoder is first trained to reconstruct past visual features (frozen encoder + fixed projection) from the current observation, the instruction and the EM bank, and is then frozen. Its reconstruction residual is used as the regression target for the TTT readout (minus the readout at phi_0), trained jointly with the flow-matching action loss. The fast weights are written with visual tokens keyed by spatiotemporal address. On a pi0.5 backbone, the method improves EM baselines by 2.2-7.1 points on RoboMME (2 EM variants x 2 integration designs), by 5-6 points on one RMBench configuration, and improves task progress on three real-world tasks.

Critical strengths
  • Asking what a fixed-capacity memory should store when an explicit bank already exists is a reasonable question. The method plugs into different EM designs without changing them.
  • Gains are consistent across the four EM/integration configurations on RoboMME, and the real-world evaluation includes a ~3-minute, 8-stage task.
  • The fast-weight update runs once per policy call and is reused across flow steps, so the latency overhead over EM is moderate (173 -> 227 ms).
Critical weaknesses
  1. Limited novelty. Combining an explicit/attention store with a test-time-updated neural memory is already the design of Titans/Atlas, and RoboTTT and WAM-TTT already use TTT as memory in robot policies. What is new here is one auxiliary target for the TTT readout. That would be enough if the paper showed this target does what is claimed, but it does not (points 2-4).

  2. The objective does not match the claim. The paper says the fast weights keep "task-relevant history not recoverable from the current context". The residual, however, is an L2 error in frozen visual-feature space, and nothing in it is task-specific. Feature residuals are likely dominated by content the decoder cannot predict and the policy does not need (arm configuration at step s, lighting, texture, background), and a deterministic L2 decoder also leaves irreducible variance in the residual. Task relevance only comes in through the action loss, which plain EM+TTT also has. Also, the write path (Eq. 9-10) stores the full feature y, not the residual. The "residual" only appears in the readout supervision, so the claim that the fast-weight state is specialized to complementary information is not supported.

  3. The key ablation is missing. The gain could come from any history-reconstruction auxiliary loss on the TTT pathway, with the decoder playing no role. The necessary control is the same pipeline with r = y, i.e. reconstructing the full history without the decoder. Fig. 3(a) only compares lambda_res = 0 against lambda_res > 0, which cannot separate the two explanations.

  4. The central thesis is never tested directly. The motivation is that a bounded EM drops information, but the memory budget K and the episode length are never varied. If Residual Memory compensates for eviction, its gain should shrink as K grows and increase with horizon. There is also no analysis of what the decoder fails to reconstruct or what delta encodes. For example, the paper could probe delta vs. EM tokens for task state (count, completed stage, identity of occluded objects), or check whether the residual is large on events the EM evicted.

  5. Small effects and unclear training parity. The best RoboMME result goes from 42.8 to 45.4, and several per-suite gains are within the reported std (e.g. Permanence, FrameSamp w/ Expert: 24.6 -> 25.6). TTT-RM gets two additional training stages, including full-policy fine-tuning, and it is not stated whether the EM baselines were trained for the same number of extra steps. Fig. 3(c) shows Stage 3 alone already accounts for part of the gain. RMBench uses a single configuration. The real-world results use 20 trials per task with no confidence intervals. The real-world EM baseline (records of observation, action and outcome) differs from the simulation EM, and the paper does not say which EM TTT-RM is built on in the real-world setup.

Questions for authors
  1. What are the results with r = y (no decoder), with everything else unchanged?
  2. Fig. 3(a) at lambda_res = 0 and Fig. 3(b) "EM+TTT" both seem to describe TTT added to EM without residual supervision, but they report different FrameSamp numbers (40.0 vs 42.8). What is the difference between the two settings?
  3. The full TTT-RM numbers in Fig. 3 (38.4 / 46.6) do not match Table 1 (38.0 / 45.4) for the same Modul configurations. Which is the main setting?
  4. Is the decoder trained on the same demonstrations as the policy? If it fits the training trajectories well, the residual on training data approaches zero and the supervision becomes weak. Please report train/held-out reconstruction error.
  5. How does the gain change with K and episode length?
  6. Did the EM baselines receive the same additional fine-tuning budget as Stages 2+3?
  7. What does +/- denote (seeds or evaluation episodes)? Were the TTT baselines retrained or taken from the RoboMME paper?
  8. The training-cost comparison with RoboTTT (7.5 vs. ~110 GPU-hours): do both use the same data and the same initialization?
7 comments
AuthorsOpenReview ↗

Rebuttal by Authors (3)

W4 & Q5. Memory capacity, history dependence, and residual information

We conducted additional experiments varying Explicit Memory (EM) capacity and task length, alongside analyses of decoder reconstruction errors and Residual Memory (RM) readouts. The results support RM's complementary role: its performance advantage decreases as EM capacity increases, while its readouts recover additional historical information useful for action prediction.

1. Effect of Explicit Memory capacity. We evaluate FrameSamp w/ Modul on RoboMME with memory budgets K∈K\in{256,512,1024} (default: K=512K=512), using matched training budgets and identical evaluation episodes to ensure controlled comparisons.

Table R3. Effect of EM capacity (success rate, %).

KKEMTTT-RMImprovement
25634.7 ± 0.737.5 ± 0.6+2.8 pp
51242.8 ± 1.945.4 ± 1.0+2.6 pp
102444.4 ± 1.445.9 ± 0.8+1.5 pp

TTT-RM outperforms EM at all three capacities, with its advantage decreasing from 2.8 to 1.5 percentage points as KK increases. This trend supports our motivation that RM provides greater complementary benefit when explicit historical coverage is limited.

2. What the decoder misses and RM readouts recover. To assess how EM coverage affects reconstruction, we evaluate the history decoder on 256 held-out episode contexts, comparing historical patches fully retained in EM against those absent from EM. At K=512K=512, normalized reconstruction error is 1.75× higher for FrameSamp and 1.94× higher for TokenDrop on absent patches, indicating that historical information not retained in EM is more difficult to reconstruct.

We next examine whether RM readouts recover information unexplained by the decoder. Under TokenDrop, incorporating RM readouts reduces historical-feature reconstruction MSE by 2.80%. Figure 6 (Appendix D) visualizes these improvements across representative ButtonUnmaskSwap and PickXtimes rollouts, showing stage-dependent improvements in regions associated with button pressing, container unmasking, and cube manipulation. These suggest that RM recovers historical features associated with task interactions.

To assess their relevance for control, we conduct expert-action prediction probing on 93 held-out episodes using instruction-matched shuffled controls. Correct RM readouts reduce action-prediction MSE by 4.76% relative to context alone and 12.52% relative to shuffled readouts from episodes with identical tasks and instructions. Together, these results indicate that RM improves reconstruction of task-associated historical features while providing additional episode-specific information useful for action prediction.

3. Effect of episode length. To examine performance under increasing history demands, we vary the required repetitions in PickXtimes (X∈1,3,5X\in{1,3,5}) and SwingXtimes (X∈1,2,3X\in{1,2,3}) at K=512K=512, using matched initial scenes within each task.

Table R4. Success rates (%) across task lengths.

TaskRepetitionsFrameSamp (EM → RM)TokenDrop (EM → RM)
PickXtimes1100 → 10095 → 95
395 → 9565 → 75 (+10 pp)
575 → 7545 → 50 (+5 pp)
SwingXtimes195 → 95100 → 100
295 → 9595 → 100 (+5 pp)
385 → 8575 → 85 (+10 pp)

Under TokenDrop, RM provides no gain at the shortest repetition counts but improves success rates by 5–10 percentage points on longer sequences, whereas FrameSamp shows no additional gains. These results suggest that longer sequences can reveal additional benefits from RM, depending on the underlying EM design rather than task length alone.

Taken together, these findings support RM's role as complementary historical memory. Its numerical advantage decreases as EM capacity increases, while its readouts recover task-associated historical features and provide additional episode-specific information useful for action prediction.

AuthorsOpenReview ↗

Rebuttal by Authors (4)

W5 & Q6. Experimental fairness and robustness

Thank you for raising these concerns. We conducted additional experiments to address training parity, extend the RMBench evaluation, and quantify uncertainty in real-world performance. The results show that TTT-RM's improvements persist under matched training budgets and across different memory configurations, providing further evidence that its benefits are not simply due to additional fine-tuning.

1. Training parity (Q6). The original EM baselines did not receive the additional training updates used in TTT-RM's Stages 2–3. To directly assess whether this accounts for the performance improvements, we continued training the original EM baselines for 1,500 additional updates, matching TTT-RM's additional optimization budget and using the same training data.

Table R5. Effect of matched additional training on RoboMME (success rate, %).

MethodEMEM (+1,500 updates)TTT-RMGain over extended EM
FrameSamp w/ Modul42.843.746.6+2.9 pp
TokenDrop w/ Modul30.931.938.4+6.5 pp

Additional fine-tuning improves EM by 0.9–1.0 percentage points, whereas TTT-RM retains advantages of 2.9 and 6.5 percentage points over the extended baselines. These controlled comparisons show that additional optimization alone cannot explain TTT-RM's gains, supporting the contribution of the proposed Residual Memory mechanism.

2. Consistency of improvements and additional RMBench evaluation. Although some individual suite-level gains on RoboMME are modest, TTT-RM improves over the corresponding EM baseline in all 16 comparisons across four memory configurations and four task suites, with average success-rate gains ranging from 2.2 to 7.1 percentage points. We further extend RMBench evaluation beyond FrameSamp w/ Modul by introducing a TokenDrop w/ Modul configuration.

Table R6. RMBench results across EM configurations (success rate, %).

ConfigurationM(1): EM → TTT-RMM(n): EM → TTT-RM
FrameSamp w/ Modul30.5 → 36.5 (+6.0 pp)10.8 → 15.8 (+5.0 pp)
TokenDrop w/ Modul31.2 → 36.4 (+5.2 pp)9.7 → 12.0 (+2.3 pp)

TTT-RM improves performance under both EM configurations and memory-complexity settings, providing additional evidence that its benefits extend beyond a single memory implementation.

3. Real-world evaluation and EM configuration. We clarify that TTT-RM and its corresponding EM baseline use identical underlying memory implementations within each experimental setting. In simulation, EM retains selected historical visual representations. In real-world experiments, both methods use the same episodic bank of up to 72 observation–action–outcome records, with FIFO eviction and learned cross-attention retrieval.

Real-world performance is measured using a stage-based task-progress score, which awards partial credit for completed substeps (100 indicating full completion) and penalizes unnecessary actions beyond the instructed goal where applicable. This metric captures how far each rollout progresses rather than only whether the entire task succeeds. We additionally quantify uncertainty using 95% confidence intervals for differences in mean task progress.

Table R7. Real-world mean task progress score (0–100) and improvements.

TaskStrongest baselineTTT-RMImprovement (95% CI)
Pen to cabinetBase: 55.565.5+10.0 [−18.3, 38.3]
Pick N pensRoboTTT: 47.073.5+26.5 [4.0, 49.0]
Stow objectsEM: 50.577.0+26.5 [2.8, 50.3]

TTT-RM achieves higher mean task progress across all three tasks, with larger improvements on the more memory-demanding tasks. The relatively simple Pen to cabinet task shows a 10.0-point improvement, whereas Pick N pens, which requires tracking completed transfers, and Stow objects, which requires maintaining task progress across eight stages, both achieve 26.5-point improvements with positive 95% confidence intervals. The stage-wise progress profiles further show stronger progression through successive task milestones. This pattern is consistent with RM providing greater benefits when successful execution depends more heavily on historical information, supporting its effectiveness for long-horizon, memory-dependent manipulation.

Overall, these additional evaluations help to strengthen the empirical evidence. The matched-budget results show that extended training alone does not account for TTT-RM's improvements, while the additional RMBench configuration and real-world uncertainty analysis provide a clearer assessment of its performance across memory settings and tasks.

AuthorsOpenReview ↗

Rebuttal by Authors (5)

Q2. Difference between λres=0\lambda_{\mathrm{res}}=0 and EM+TTT

The two settings differ in whether the TTT reader receives residual-supervised preparation before policy fine-tuning. In Fig. 3(a), λres\lambda_{\mathrm{res}} is varied only during Stage 3, after all variants have undergone residual-supervised preparation in Stage 2. Thus, λres=0\lambda_{\mathrm{res}}=0 removes residual supervision during final fine-tuning but retains its effects from Stage 2, yielding 40.0%. In contrast, the EM+TTT baseline in Fig. 3(b) is trained without residual supervision at any stage, achieving 42.8%.

To clarify this distinction and further examine the role of Stage 2 preparation, we conducted an additional controlled sweep without residual-supervised reader preparation, using matched initialization, training data, and policy-training schedules.

Table R8. Effect of residual-loss weight without reader preparation (FrameSamp w/ Modul).

λres\lambda_{\mathrm{res}}00.251416
Success rate (%)42.845.645.444.746.6

Under this matched protocol, λres=0\lambda_{\mathrm{res}}=0 reproduces the EM+TTT baseline at 42.8%, confirming that the original discrepancy reflects different training histories. The 40.0% result in Fig. 3(a) also shows that Stage 2 preparation alone does not improve subsequent action-only fine-tuning. Residual supervision remains effective without Stage 2, with all positive loss weights outperforming EM+TTT. However, achieving the best result (46.6%) requires λres=16\lambda_{\mathrm{res}}=16 without preparation, compared with λres=1\lambda_{\mathrm{res}}=1 or 44 with preparation. These findings suggest that Stage 2 reduces sensitivity to the residual-loss weight and the need for extensive tuning, while the residual objective drives the performance gains.

Q3. Numerical differences between Fig. 3 and Tab. 1

Tab. 1 reports our main benchmark results, averaged across three evaluation seeds with standard deviations, whereas Fig. 3 reports single-seed results using the same fixed evaluation seed for all methods, including the EM baselines. Specifically, Tab. 1 reports 38.0 ± 0.5% and 45.4 ± 1.0% for TokenDrop w/ Modul and FrameSamp w/ Modul, respectively, while Fig. 3 reports 38.4% and 46.6% under the fixed seed. These differences arise from evaluation-seed variability rather than changes in the model architecture or training configuration. Importantly, all ablation variants are evaluated under the same fixed seed, ensuring consistent evaluation conditions and fair comparisons across design choices.

Q4. Decoder training data, generalization, and residual supervision

Thank you for raising this concern. The history decoder is trained on the same demonstration corpus as the policy, with 1,280 episodes for training, 160 for checkpoint selection, and 160 held out for evaluation. To examine whether decoder fitting leads to overfitting or ineffective residual supervision, we evaluate the frozen decoder on training and held-out demonstrations using normalized reconstruction MSE (feature MSE divided by target mean-square energy).

Table R9. Decoder reconstruction error on training and held-out demonstrations.

MemoryTraining MSEHeld-out MSERelative increase
FrameSamp0.0015900.0016695.00%
TokenDrop0.0025630.0026653.97%

The held-out errors are only 5.0% and 4.0% higher than the training errors, respectively, indicating modest generalization gaps. The low overall reconstruction errors reflect the decoder's ability to recover much of the historical feature information from the existing context. Importantly, the remaining residuals are larger where historical information is missing from EM. As detailed in W4, held-out reconstruction errors are 1.75× higher for FrameSamp and 1.94× higher for TokenDrop when historical patches are absent from EM rather than fully retained.

We further verify that these residuals provide an effective learning signal. In our controlled FrameSamp experiment without Stage 2 preparation, introducing residual supervision improves success from 42.8% at λres=0\lambda_{\mathrm{res}}=0 to 45.6% at λres=0.25\lambda_{\mathrm{res}}=0.25, with the best evaluated setting reaching 46.6%. Thus, despite the small overall reconstruction error, residual supervision produces measurable improvements in downstream control.

Together, these results show that the decoder generalizes well while leaving residual information associated with gaps in EM coverage, and that supervision from these residuals improves policy performance. This supports the intended role of the decoder in identifying complementary historical information rather than simply reconstructing the full history.

Reviewer_VLXrOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper introduces TTT-RM, which has TTT serve as Residual Memory to preserve task-relevant history which cannot be extracted from the policy’s current observation and memory. TTT-RM is compatible with designs of external memory modules as well. It performs better on simulated benchmarks such as RoboMME and RMBench and has a real-world task which is 3 minutes long with multiple substages. The TTT supervises the mechanism with resulting residuals for history unexplained by this context used for fast-weights.

Critical strengths

There could be more real-world experimentation but it is tested on simulation benchmarks that are considered not saturated and useful for the field.

Critical weaknesses

None

Questions for authors

Why design it as residuals and not make it save absolute entries?

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of Author–Reviewer Discussion

Dear Area Chair and Reviewers,

We sincerely thank you for your time and constructive feedback throughout the review process! We appreciate the reviewers' recognition that our rebuttal has addressed most of the original concerns, substantially strengthened the empirical evaluation, and clarified important aspects of our formulation.

As the discussion period draws to a close, we would like to highlight the key contributions and additional evidence.

1. Distinct methodological contribution. TTT-RM addresses a fundamental memory-learning question: what should fast-weight memory learn when the policy already has access to bounded Explicit Memory (EM)? Rather than simply combining existing memory mechanisms, we introduce a context-conditioned residual learning objective that trains fast-weight readouts to recover historical information unexplained by the current observation and EM. This establishes a distinct approach to learning complementary memory for history-dependent robot control.

2. Stronger controlled experimental evidence. We conducted extensive additional experiments, including independently tuned absolute-history reconstruction baselines, matched EM fine-tuning, unified three-seed evaluations, memory-capacity and task-length studies, expanded RMBench comparisons, and real-world uncertainty analyses. Importantly, residual supervision outperforms absolute reconstruction even after independent coefficient tuning, demonstrating that the specific reconstruction target matters beyond generic auxiliary supervision.

3. Direct validation and interpretation of Residual Memory. We further analyzed decoder reconstruction errors, historical features recovered by RM readouts, and their action-predictive information, complemented by qualitative visualizations. These analyses provide direct evidence of residual feature reconstruction and action-predictive content, while clarifying that our method targets information unexplained by the available context, rather than simply information evicted from EM.

Together, these contributions establish a distinct memory-learning formulation, demonstrate the effectiveness of residual supervision through controlled comparisons, and validate its applicability across multiple memory architectures and robotic tasks. We again sincerely appreciate the reviewers' thoughtful suggestions, which have helped us substantially strengthen the paper.

Best regards,

The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.