LOOPED TRANSFORMERS LEARN ONE OF TWO ALGORITHMS, AND TRAINING MOVES THE ODDS

A looped transformer applies the same block many times over. On a task with a known depth requirement, computing every running product of a sequence of group elements, such a model could in principle work in logarithmic depth. Trained on one, it instead reads the sequence one element per loop, and a growing literature asks what would make it do otherwise. We answer that by intervening rather than by timing: overwriting the model’s internal state at one position and one loop, and asking which outputs change. Doing this across two independent sets of checkpoints, 237 and 90, shows that these models do not learn a spectrum of more or less efficient circuits. They learn one of two algorithms, and the distribution of their measured speed is bimodal, with a slow mode sitting exactly at one position per loop and a fast mode near three. The structure survives a test that assumes no parametric form, and survives refitting on one checkpoint per training contract. Training therefore controls the odds of getting the better algorithm, not the algorithm’s quality. Making the efficient one available does not move those odds: un-tying the weights leaves the median slope at exactly 1.00, as does conditioning each loop on its index, as does supervising the efficient circuit’s own intermediate quantities, which hands the model the target computation. Only making the slow algorithm infeasible moves them. Since the achievable speed varies from seed to seed while each model faces a hard threshold, the population curve is smooth: we registered a sharp critical budget, found none, and report that as a failure. Holding the depth budget proportional to length, reach does not fall from n=64 to n=128, so length is not the binding variable; capacity is, rising from 17 of 40 solved at 5M parameters to 11 of 12 at 50M (p = 0.003). It does not buy speed. What it buys is fidelity at matched depth and a shift in which algorithm is installed, and we measure both. This is one family of synthetic tasks, models up to 50M parameters, and a causal measurement that reaches n ≤ 32. A frontier is only measurable against a known depth requirement, which natural text does not supply; on the two public looped LMs we can therefore only test behaviour, and there depth from 1 to 64 does not move a two-composition ceiling. Within that scope every claim was registered before the run that tests it, which is what makes the negative results usable: the capability result is believable because it was a prediction rather than a summary. Every registered prediction is scored in the appendix, including those that failed.