Statistical Gödel Machines: Safe Self-Evolution under Irreversible Updates

Machine learning systems are increasingly embedded in iterative pipelines in which accepted changes to models, training procedures, or optimization strategies persist and influence subsequent updates. When such changes are costly or effectively irreversible, reliable self-improvement requires more than selecting the candidate with the best noisy validation estimate: the risk of erroneous commits must be controlled over the entire update trajectory. We formulate this setting as risk-controlled recursive self-modification and introduce the Statistical Gödel Machine (SGM), a decision layer that accepts an update only when its improvement is statistically certified. SGM allocates a summable global error budget across irreversible commits, yielding an explicit bound on the probability of ever accepting a non-improving update, including under adaptively generated proposals. We further introduce Confirm-Triggered Harmonic Spending (CTHS), which allocates risk to confirmation events rather than every proposal, improving statistical power when costly commit decisions are sparse. Across supervised learning, reinforcement learning, and optimization benchmarks, SGM filters apparent improvements caused by evaluation noise while retaining power to accept reproducible gains. In a 40-iteration ImageNet-100 self-modification trajectory, SGM produces sparse certified updates and stable incumbent performance while requiring substantially less compute than confirming every proposal. These results suggest that persistent self-improvement is naturally viewed as a trajectory-level statistical decision problem, with optimization generating candidate updates and statistical certification governing which updates are committed.