acceptodds
Under review as a conference paper at ICLR 2027

Looped Transformers under the Jacobian Lens: Does the Global Workspace Survive Recurrence?

Abstract

A Jacobian lens identifies verbalisable, causally editable representations in standard transformers, but it is unclear whether the same workspace functions arise when depth reuses weights. We extend the lens to recurrent architectures by virtual unrolling and fit distance-controlled lens families for Ouro-2.6B and Huginn-0125, with Qwen3.6-27B as an untied baseline. Readouts, sparse concept inventories and eleven behavioural experiment families reveal two maintenance regimes. Ouro reconstructs content across supervised loop checkpoints: sustained writes outperform single-loop writes, while recovery from ablation depends on its width. Huginn's content remains readable across sixteen recurrences, but input-path interventions show that this persistence depends on continued re-injection; reads and effective edits follow a roughly two-recurrence horizon. Reporting existing and newly injected content also dissociates: Huginn excels at the former yet fails at the latter, which only Ouro achieves among the tested checkpoints. The results support functional workspace organisation under recurrence while distinguishing model behaviour from limitations of the readout instrument.

open until 14 Dec 2026

est. 32% chance this paper gets accepted at ICLR 2027.

Reject 68%Accept 32%

What do you think this paper will get?

All positions stay anonymous.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.