When Do Two Hops Suffice? Learning Reusable Composition in Looped Transformers

Learning individual operations does not guarantee reliable composition. We study a synthetic task in which each operation maps one entity to another. Training uses all single-step examples and selected two-step answers, together with state targets derived from the same shallow labels, random input prefixes, and a self-supervised loss encouraging waiting-state preservation on constructed states. With one loop per operation, the resulting full-causal looped Transformer achieves 100% accuracy at tested depths of 256, 512, and 1024 and 99% at depth 2048. State interventions show that intermediate states can decode correctly yet fail as inputs to the next operation. These results show how learning to preserve usable states supports deep composition from shallow answer supervision.