- Rating
- 3(+1)
- Confidence
- 4
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper proposes TTT-RM, a framework that combines Explicit Memory (EM) with Test-Time Training to retain residual historical information that is not recoverable from the current observation and retrieved memory. A history decoder reconstructs past visual representations from this context, and its reconstruction residual supervises the change in TTT readouts relative to the episode's initial state. Incoming visual features update the fast weights during each episode, while action and residual objectives jointly train the slow parameters governing memory writing and retrieval. The method is evaluated on RoboMME, RMBench, and three real-world manipulation tasks, with additional ablations and latency analysis.
Critical strengths
- The paper provides a precise and clearly presented formulation of memory-conditioned action policies, Explicit Memory (EM), and Test-Time Training (TTT). Especially, the distinction between online fast-weight updates and outer-loop learning, together with the staged training procedure, makes the proposed framework easy to follow.
- The proposed method is comprehensively evaluated across two simulation benchmarks and real-world manipulation tasks with different memory requirements, including counting and tracking task progress. The reported improvements over EM baselines across these settings demonstrate the practical potential of combining explicit and parametric memory for history-dependent control.
Critical weaknesses
- The paper does not sufficiently establish whether failures stem from missing historical information, inadequate use of retained context, or difficulties mapping that context to low-level actions. Controlled analyses of these alternatives would better justify the need for additional historical feature reconstruction.
- The EM and TTT baseline configurations differ across RoboMME, RMBench, and real-world tasks, and some baseline details are not specified (e.g., the TTT variant used on RMBench and the real-world EM's selection rules and memory budget). These omissions limit reproducibility and make it difficult to assess performance across domains under a consistent memory setup.
Questions for authors
- Could the authors provide rollout examples identifying past timesteps and visual patches with large decoder reconstruction errors, and show how TTT readouts recover task-critical information unavailable from the current observation and EM? Furthermore, how do these examples distinguish missing historical information from decoder approximation error or task-irrelevant visual variation?
- In Figure 3(b), how would TTT-RM compare with an EM+TTT baseline trained to reconstruct full historical visual representations under the same setup? Including this baseline would clarify the specific benefit of residual targets beyond historical reconstruction supervision.
6 comments
Rebuttal by Authors (2)
W2. Baseline configurations and reproducibility
We use benchmark-specific Explicit Memory configurations to evaluate TTT-RM across different memory architectures, while ensuring that TTT-RM and its corresponding EM baseline use identical memory representations, capacities, selection rules, and retrieval mechanisms within each benchmark. This allows us to isolate the contribution of Residual Memory (RM) without confounding it with changes to the underlying EM design. We clarify the configurations below.
1. Simulation benchmarks. On RoboMME, we evaluate TokenDrop and FrameSamp under both Expert and Modul integration architectures, all with the same visual-token budget of . The standalone TTT baselines follow RoboMME's original Expert and Modul configurations. On RMBench, our main EM baseline uses FrameSamp-style memory with Modul integration and . We additionally evaluated TokenDrop w/ Modul on RMBench, where TTT-RM improves over its EM baseline from 31.2% to 36.4% on M(1) and from 9.7% to 12.0% on M(n). For the standalone TTT baseline on RMBench, we reimplement RoboTTT with a separate fast-weight MLP in each of the 18 action-expert layers, a learned write-rate gate, and gated residual integration of memory readouts.
2. Real-world experiments. EM maintains an episodic bank of up to 72 interaction records, each encoded as a 256-dimensional token representing the observation, commanded action, and resulting outcome. One record is created per action chunk and finalized when the subsequent observation becomes available. Records are retained using a first-in, first-out (FIFO) policy when the bank reaches capacity. The policy accesses all retained records through learned cross-attention, without additional keyframe selection or top- filtering. TTT-RM uses this same EM configuration and memory budget as the real-world EM baseline. The standalone TTT baseline follows our RoboTTT reimplementation described above.
Thus, although the memory representations differ across benchmarks, each EM–TTT-RM comparison holds the underlying explicit-memory configuration fixed, allowing performance differences to be attributed to the addition of RM under the respective training protocols.
Rebuttal by Authors (3)
Q1. Historical reconstruction and Residual Memory readouts
Thank you for this valuable suggestion. We conducted additional quantitative analyses and rollout visualizations to examine (1) what historical information Residual Memory (RM) recovers, (2) how these improvements relate to missing Explicit Memory (EM) coverage, and (3) whether the learned readouts provide information useful for control.
1. Historical feature recovery and rollout visualization. We compare decoder-only reconstruction with reconstruction augmented by RM readouts . Across 16,352 historical-address evaluations from 256 decisions spanning 16 RoboMME tasks, RM reduces historical-feature reconstruction MSE by 2.80% under TokenDrop. This provides direct evidence that RM readouts recover information unexplained by the decoder, supporting their role as complementary memory rather than merely an additional action-conditioning pathway.
We additionally introduce Figure 6 (revised Appendix D), visualizing patch-level reconstruction improvements across representative RoboMME rollouts. In ButtonUnmaskSwap, improvements appear around button-pressing and container-unmasking regions, with spatial patterns changing across task stages. In PickXtimes, improvements appear around cube manipulation during repeated placements and the stopping button at task completion. These task-associated patterns provide qualitative evidence that RM recovers historical features relevant to task progression.
2. Distinguishing missing history from decoder approximation error. As detailed in W1, we examine reconstruction errors according to EM coverage. At , across 256 held-out contexts, decoder reconstruction errors are 1.75× higher for FrameSamp and 1.94× higher for TokenDrop when historical patches are absent from EM than when fully retained. This association indicates that limited historical coverage contributes to reconstruction difficulty.
3. Action relevance of recovered information. To assess whether RM readouts provide action-relevant information beyond visual reconstruction, we conduct held-out action-prediction probing, comparing a fixed context representation alone, augmented with correct RM readouts, or augmented with shuffled readouts from episodes sharing the same task and instruction.
Table R2. Action-prediction probing on held-out episodes (lower MSE is better).
| Comparison | Action MSE | Reduction with correct RM |
|---|---|---|
| Context only → Context + RM | 0.09215 → 0.08777 | 4.76% |
| Context + shuffled RM → Context + correct RM | 0.10033 → 0.08777 | 12.52% |
Correct and shuffled RM readouts use the same trained probe and normalization. The improvement over context alone demonstrates that RM provides additional action-predictive information, while the improvement over instruction-matched shuffled readouts indicates that this information is specific to the corresponding episode rather than merely reflecting the task or additional feature dimensions.
Together, these analyses support RM's role in recovering complementary historical information useful for control. RM readouts recover features unexplained by the decoder, improve reconstruction in task-associated regions, and provide additional action-predictive information.
Q2. Comparison with full-history reconstruction
Thank you for this suggestion. To isolate the benefit of residual supervision, we conducted the full-history reconstruction ablation (), replacing the decoder-defined residual target () while keeping the architecture, action objective, auxiliary-loss weight, training and evaluation protocols, and other hyperparameters unchanged.
Table R3. Residual versus absolute-history reconstruction (success rate, %).
| Method | EM + TTT | Absolute () | Residual () |
|---|---|---|---|
| FrameSamp–Modul | 42.8 | 40.6 | 46.6 |
| TokenDrop–Modul | 33.1 | 32.8 | 38.4 |
Residual supervision outperforms full-history reconstruction by 6.0 and 5.6 percentage points, respectively, while absolute reconstruction slightly underperforms naive EM+TTT in both configurations. A possible explanation is that absolute reconstruction allocates limited fast-weight capacity to information already recoverable from the current observation and EM, introducing redundant supervision that may interfere with action optimization. In contrast, the decoder-defined residual target emphasizes information unexplained by the existing context, encouraging complementary memory representations. This validates our central design choice of using context-conditioned residual supervision to encourage complementary memory representations. We have incorporated this comparison into the revised ablation section.
Rebuttal by Authors (1)
Thank you for your insightful review! In response, we have conducted additional controlled ablations and analyses, clarified our experimental setups and implementation details to improve reproducibility, and revised the manuscript accordingly. We hope these additions address your concerns and welcome further discussion.
Thank you for this insightful suggestion. To investigate whether missing historical information contributes to policy errors, we conducted additional analyses examining how historical information availability affects reconstruction, action prediction, and downstream performance.
First, we examine how incomplete historical coverage affects reconstruction. At an EM budget of K=512, we evaluate the frozen history decoder on 256 episode contexts held out from decoder training. We compare normalized reconstruction errors between historical patches whose source features are fully retained in Explicit Memory (EM) and those absent from EM. Reconstruction error is 1.75× higher for FrameSamp and 1.94× higher for TokenDrop on absent patches. These findings indicate that limited historical coverage leaves information that is substantially more difficult to reconstruct from the existing context.
Second, we directly assess the contribution of historical information to action prediction. We conduct controlled historical-cue withholding experiments under FrameSamp, keeping the current observation and policy parameters fixed while removing historical cues. This increases expert-action prediction loss by 6.03% for EM and 5.80% for TTT-RM. These results demonstrate that access to historical information contributes to action prediction independently of changes to policy parameters, rather than being solely a matter of learning to map a fixed context into actions.
Third, we examine whether the benefit of RM changes with historical information availability. We evaluate FrameSamp–Modul on RoboMME with memory budgets K∈{256,512,1024}, keeping historical selection rules, training budgets, and evaluation episodes matched across methods. TTT-RM outperforms EM at every evaluated capacity, with its numerical advantage decreasing from 2.8 to 1.5 percentage points as K increases. This trend supports our motivation for complementary historical memory: RM provides greater additional benefit when EM retains less history, while its marginal contribution diminishes as more historical information becomes available through EM.
Table R1. Effect of EM capacity on RoboMME (success rate, %).
Together, these findings identify limited historical access as a meaningful contributor to history-dependent control errors. Historical information absent from EM is harder to reconstruct, removing historical cues degrades action prediction under a fixed policy, and RM's marginal benefit decreases as EM capacity increases. These results motivate complementary historical reconstruction without assuming that missing history accounts for every failure. We further examine what RM readouts recover and their action relevance in our response to Q1.