Learning from Recurrence: Internalizing Depth-Induced Preferences in Looped Language Models

Recurrent language models can improve predictions by repeatedly applying the same parameters, making depth a property of computation rather than model size. We ask whether behavior exposed by additional recurrence can be written back into the shared computation itself, a transfer we call preference write-back. Trajectories generated by deeper execution then self-distill only the shared recurrent block, without an additional correctness signal. On a 1B model retrofitted with recurrent depth, this update substantially improves both one-pass and full-depth execution on mathematical reasoning, across models trained for mean depths from 4 to 32: GSM8K accuracy at a model's own depth rises from 56.2% to 62.7% at mean depth 4, with comparable gains at 8 and 32. Moreover, updating the block through one recurrent pass consistently outperforms training on the same targets through the full recurrent depth, while requiring less compute and memory; the resulting update also remains useful when the block is reused at later passes. A controlled comparison of looping recipes shows that this advantage of shallow training is a property of how the recurrent model was trained, not of weight tying alone. These results show that recurrent depth is not only an inference-time resource: it can generate self-supervision that improves the shared operator in both single-pass and multi-pass execution.