Six Requirements for Inference-Time Self-Iterating Models: Representation, Uncertainty, and Local Credit

Inference-time self-iteration lets a model run an iterative learner on its context before it answers. For multi-branch in-context relations—how different two items are along an attribute the model was never trained on, or compressive phase retrieval—a lifted inner loop (per-example global inversion, multipliers, local least squares) provably reaches what test-time training's gradient update, fixed-feature closed-form heads and additional layers cannot. Whether a pretrained model can host such a loop depends on its representation. We prove six requirements. Two concern the representation and hold for every learner: the task must be decodable from the base, which sets an explicit error floor floor(ν) in terms of the base's undecodable fraction ν (R1), and it must be identifiable from the context, which ties the task dimension to the context size and calls for posterior mixtures near the threshold (R2). Two concern uncertainty: inversion and readout must be consistent with the base's residual noise (R3), and in-context noise estimates must be corrected for degrees of freedom (R4). Two concern local credit: the multiplier leak must vanish on exactly decodable bases and grow with the noise (R5), and the knobs that control branch changes must be selected, not trained by backpropagation (R6). R1 and R2 give measurable diagnostics of a base. On word vectors (GloVe), a molecular graph network (GINE) and an image code (PCA), which satisfy them, the matched self-iterating learner beats gradient test-time training, quadratic closed-form heads and Transformers, with a second-seed replication and paired tests; on a base that violates decodability (MiniLM word vectors) every learner, including an oracle probe, stays above the proven floor.