acceptodds
Under review as a conference paper at ICLR 2027

Why Is Recursive Self-Improvement Hard to Verify?

Abstract

An AI system can produce better models without becoming better at improving itself. We study what evidence can establish recursive self-improvement (RSI) in AI training, where candidate updates are selected through experiments rather than authorized by proof. For a fixed target and declared system boundary, we give a sufficient finite-horizon criterion that combines target gains, causal benefits from installing self-produced updates, and more competent successor mechanisms. Those successors must also be installed and perform the next update. We derive statistical tests for these effects and a cumulative-gain bound under bounded utility and direct continuation. We also identify when the available records cannot support a reliable verdict. Analyses of public traces and a model-coded comparison of 17 benchmark families show why more complete records need not establish that an improvement mechanism has become better. The framework clarifies what must be measured to distinguish successful search from verified recursive improvement.

Then back it, or bet against it.

Related papers

Open the market on this paper to see 7 more related papers.