acceptodds
Under review as a conference paper at ICLR 2027

Recurrent Refinement Moves Candidate-Quality Readability Earlier and Stabilizes Its Coordinates

Abstract

A looped language model applies the same layer stack several times to a token position, so “physical layer 8” is not one event but one per pass. This makes a question well-posed that a feed-forward stack cannot ask: when during recurrent refinement does a property of the computation become linearly readable at a given layer, and is it carried by the same coordinates throughout? We answer it for candidate-quality information in a frozen 2.6B looped transformer (Ouro), with small bias-free antisymmetric taps selected from a fixed domain-subset sweep and evaluated on a task-disjoint five-domain candidate-ranking target under an audited pairwise protocol. Two evaluation modes separate two questions. Locus-local refit asks whether the information is readable at a loop/layer cell: held-out macro top-1 at physical layers 8 and 16 is weak on the first pass (0.410, 0.440; chance 0.282) and statistically tied with the established late-layer basis by loops 3–4 (0.633, 0.628), with loop 4 − loop 1 contrasts of +0.19 to +0.21 at every tested layer. Frozen cross-loop transplant asks whether it is carried by functionally stable readout coordinates: taps fitted at loop 1 lose 0.09–0.23 when applied unchanged at loop 4, while taps fitted at loops 2–3 transfer at parity (|Δ| ≤ 0.024). This indicates a first-pass coordinate change followed by stable frozen-transfer behavior. Both signatures replicate on three further Ouro checkpoints, and the archived RLTT taps transfer frozen across checkpoints within 0.028 of locally refit taps; the depth trend replicates qualitatively in an out-of-family depth-recurrent model (Huginn, +0.107 from step 1 to step 8), where no comparable immediate first-pass transfer discontinuity is detected. A non-looped control shows that a same-class readout exists without recurrence, so we claim a property of where and when the signal is readable in looped models, not that looping creates it. All results are readout-only; no intervention is performed.

open until 14 Dec 2026

est. 32% chance this paper gets accepted at ICLR 2027.

Reject 68%Accept 32%

What do you think this paper will get?

All positions stay anonymous.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.