acceptodds
Under review as a conference paper at ICLR 2027

Teach Once, Execute Repeatedly: Test-Time Training in Looped Transformers

Abstract

Test-time training (TTT) changes a model for the current task; looped Transformers reuse the same block for more steps. We study their composition: an update learned from demonstrations is placed inside a tied loop and executed at every recurrent visit. This placement changes its finite- horizon effect. In a linear recurrence, a rank- update reused for visits can induce an operator difference of rank , and we show this is tight: exactly reproducing it with a single-use linear update requires rank . For nonlinear networks, we use complete visit gating to recover interaction orders one through . Test-Time Compiled Loops (TTCL) learns a low-rank adapter from demonstrations and shares it across the loop. On NumSeqBench, TTCL reaches 93.3% exact-match accuracy on held-out arithmetic rules, compared with 0% when the same update is used once. In a semantic-state audit, a rank-one update reaches executed rank 16 in a 17-dimensional state and remains exact for 32 visits. On Ouro-1.4B and Ouro-2.6B LoopLMs, shared execution improves answer NLL over the same adapter used once while approaching a depthwise adapter with four times as many task parameters. Visit-gating interventions at expose high-order hidden interactions and sensitivity to visit order. Random norm-matched adapters produce larger interactions without improving NLL, indicating that the reuse gain depends on the learned update.

open until 14 Dec 2026

est. 32% chance this paper gets accepted at ICLR 2027.

Reject 68%Accept 32%

What do you think this paper will get?

All positions stay anonymous.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.