acceptodds
Withdrawn from ICLR 2027

In-Context Test Time Training for Adaptive Robot Policy Learning

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

During deployment, robots continually receive new observations that can inform subsequent actions. Yet many robot policies operate with fixed parameters and a short observation history, limiting their use of accumulated experience. Learning from this experience requires both selecting useful information and training the policy to act with a memory that changes over time. We introduce In-Context Test Time Training (In-Context TTT), a framework that combines adaptive fast-weight memory with selective writing for robot policy learning. TTT layers encode incoming observations into fast weights, while a learned gate controls the strength of each memory update. To train this recurrent adaptation process, we develop Trajectory-Consistent Adaptation (TCA), which supervises action prediction along the fast-weight trajectories generated by ordered demonstrations. After chunk-level initialization, TCA carries fast weights across an entire episode, accumulates supervised gradients at successive adaptation states, and updates the slow parameters once at the episode boundary. This aligns the memory evolution used in training with deployment, where fast weights change while slow parameters remain fixed. On the six longest RoboCasa GR-1 tasks, selected by an average demonstration length of at least 500 steps, In-Context TTT achieves 53.3% mean success, compared with 50.0% for No-update SFT and 47.0% for the original policy. Real-robot evaluations obtain 7/10 successes in banana picking, 6/10 in cup stacking, and 4/10 in arranging digit-shaped blocks. Together, selective writing and trajectory-consistent supervision provide a way to learn from interaction history using demonstration data. Our core implementation is available at https://anonymous.4open.science/r/In-Context-TTT-E538.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Robot Learning, In-Context Learning, Test-Time Training

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_Ei9yOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper adopts test-time training for allowing robotic control models to have weights that can be adapted to allow greater effective context lengths. Concretely, the proposed method adds a learned write gate to fast-weight memory and trains it so that the policy can selectively remember and use memory over long episodes.

Critical strengths
  1. The design choices are reasonably well-explained. Section 3 is particularly well-written where each component is well-motivated and then the final algorithm is presented in a clear manner.
  2. The method is SFT-only and does not require RL or other on-policy training methods, which is impressive since the latter are often highly expensive for robotic tasks.
Critical weaknesses
  1. The scope of novelty is dubious to me. The proposed algorithm’s novelty comes from fine-tuning TTT layers in the vision-language-action model on ordered full episodes instead of independent chunks. Other aspects such as the one adaptation rule per episode does not seem particularly novel to me since it is essentially accumulating gradients over the episode and stepping once, which gives the same gradient as a full episode loss.
  2. Extremely poor choice of baselines. For the results in Table 2, the only baseline is the base pretrained Karos world-action model and the no-update SFT model. For the real robot experiments (table 4), there is no baseline comparison whatsoever. There’s a wide variety of baselines that one can compare against: these can be open-source models such as RoboTTT, openVLA, CosmosPolicy, MEM, etc. Similarly, the paper requires significantly more ablations. For example, TCA trained with no gate during training as well as deployment, gate trained without TCA, as well as other methods for increasing context lengths of models (PTP [1], keyframes [2], etc.)
  3. Performance gains are extremely marginal in table 2, the gain of 3.3 points is within the standard error. Same for the results in table 3.
  4. None of the results show confidence intervals or standard deviation in performance on multiple training seeds.
  5. Table 3 shows that when gt=0g_t=0, the performance barely changes, in which case the importance of the gate is under serious doubt to the reader.
  6. Length of demonstrations is not a reliable measure of whether or not the tasks require long-range memory. Most of the tasks in the benchmarks considered have optimal policies that are highly Markovian: the most recent observation captures most of the important information required for solving the task. It would be useful to consider more memory-demanding tasks such as those in [1] or [2] or simple tasks where objects are momentarily occluded for example.

[1] Past-token prediction. https://arxiv.org/pdf/2505.09561 [2] Mem. https://arxiv.org/pdf/2603.03596 [3]. pi 0.5. https://www.pi.website/download/pi05.pdf

Questions for authors

See weaknesses.

Reviewer_cpxvOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces gated test-time-training (TTT) layers into the pretrained Kairos action policy to maintain fast-weight memory across action chunks. It proposes a two-stage training procedure: no-update SFT followed by Trajectory-Consistent Adaptation (TCA), which carries fast weights across chunks during training.

Critical strengths
  1. Training on independent chunks does not reflect deployment with persistent fast weights. TCA addresses this by carrying the state across chunks during training. The implementation details and released code may also be useful for future work on memory in diffusion-based policies.
  2. The paper specifies its long-horizon task selection rule, lists the candidate tasks, and reports task-level regressions rather than only aggregate results.
Critical weaknesses
  1. The empirical gains are not sufficiently convincing. The improvement from no-update SFT to TCA is only 3.3 percentage points, with 50 trials per task and no confidence intervals or multiple-seed results.
  2. The effect of TCA is confounded by training compute. Based on the reported settings, TCA processes roughly 7–8 times as many chunk-level losses as no-update SFT. Without a compute-matched SFT baseline, the gains cannot be attributed to trajectory-consistent training rather than additional optimization.
  3. The training objective does not clearly support the claimed memory behavior. Since fast weights are detached between chunks, the writer and gate receive no gradient through the effects of a write on later actions. It is therefore unclear how the model learns to preserve information that is useful for future chunks.
  4. The contribution of the gate and episode-level update rule remains unclear.
  5. The need for persistent memory is not established. The evaluation lacks comparisons with simpler history baselines, such as frame stacking or longer attention context, and does not establish that the selected tasks genuinely require information from earlier observations.
Questions for authors
  1. What does “matched demonstration exposure” mean in terms of chunk-level losses? Can the authors compare TCA with no-update SFT trained for the same compute budget?
  2. Why does the Placemat result differ between Tables 2 and 3? Are these results from the same checkpoint and evaluation seeds? Please report confidence intervals and results across multiple seeds.
  3. Which denoising step and noise level are used to generate committed writes during training and deployment? If they differ, how is train-deployment consistency maintained?
  4. How does the gate differ from the existing learned inner learning rate? Please include a no-gate training baseline and compare episode-level accumulation with per-chunk updates.
  5. Can the authors report results on the remaining RoboCasa tasks and compare the original policy and no-update SFT against TCA on the real-robot tasks?
Reviewer_iDK3OpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

The paper studies how a robot policy can use what it observes during an episode. It adds test-time-training (TTT) layers to the Action DiT of the Kairos world-action model. Each TTT layer holds fast weights. At every new observation, the fast weights are updated by a gradient step on a self-supervised reconstruction loss. A learned gate controls how much of each update is written to memory. The fast weights persist across action chunks within an episode and reset between episodes.

The slow parameters are the writer, the gate, the readout and the action model. They are trained offline and stay fixed at deployment. Trajectory-Consistent Adaptation (TCA) is the procedure used to train the slow parameters. It runs the fast-weight update across a whole demonstration and supervises the expert action at each resulting memory state. The gradients are accumulated over the episode. The slow parameters are updated once at the end of the episode during training.

TCA starts from a model called "No-update SFT", which is trained on independent chunks with the fast-weight updates switched off. It therefore only ever sees the initial memory. On the longest RoboCasa tasks, TCA improves modestly over No-update SFT. No-update SFT improves over the original policy (the baseline)

The paper also overrides the gate at test time. It tries the learned gate, a gate that never writes, and a gate that always writes. Success changes very little across these three settings. Finally, the paper reports three real-robot tasks and a preliminary study on another backbone that compares attention with TTT mixers.

Critical strengths
  1. The idea behind TCA is sound. Independent-chunk training only exposes the policy to its initial memory. At deployment the policy sees many evolved memory states, and TCA targets this gap directly.
  2. The authors evaluate one trained checkpoint with the gate forced to never write or always write, which is a good probe of memory use. The authors also note honestly that the learned gate does not help everywhere.
Critical weaknesses
  1. TCA improves only modestly over No-update SFT. The same checkpoint gives a noticeably different result on one task (Placemat) in the main results and in the gate experiment, so the evaluation seems a bit noisy. Please confirm whether evaluation seeds are fixed and why this task differs.
  2. The paper says the stages are compared at matched demonstration exposure, but it never defines what this means. No baseline continues No-update SFT with comparable extra training, so the gain may come from more training rather than from carried memory.
  3. The paper does not say whether the "original policy" is the pretrained policy without TTT layers. It also does not say whether it was fine-tuned on the same demonstrations. The step from the original policy to No-update SFT is therefore hard to interpret.
  4. It is hard to conclude that the memory is used. Forcing the gate to never write barely changes success. There is also no experiment that resets the memory within an episode (for example at every chunk) or supplies memory from a different episode.
Questions for authors
  1. What exactly is the original policy? Was it fine-tuned on the same demonstrations as No-update SFT and TCA?
  2. Can you add a baseline that continues No-update SFT with the same optimizer steps and data as TCA? If TCA still wins by more than seed-to-seed variability, I would raise my score.
  3. Can you evaluate TCA with its memory reset at every chunk, or given the memory of a different episode? If success barely changes, the gain is not coming from the use of history.
  4. Can you report multiple seeds with confidence intervals? Please also explain why the same checkpoint differs on Placemat between the main results and the gate experiment.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.