acceptodds
Under review as a conference paper at ICLR 2027

Statistical Gödel Machines: Safe Self-Evolution under Irreversible Updates

Abstract

Machine learning systems are increasingly embedded in iterative pipelines in which accepted changes to models, training procedures, or optimization strategies persist and influence subsequent updates. When such changes are costly or effectively irreversible, reliable self-improvement requires more than selecting the candidate with the best noisy validation estimate: the risk of erroneous commits must be controlled over the entire update trajectory. We formulate this setting as risk-controlled recursive self-modification and introduce the Statistical Gödel Machine (SGM), a decision layer that accepts an update only when its improvement is statistically certified. SGM allocates a summable global error budget across irreversible commits, yielding an explicit bound on the probability of ever accepting a non-improving update, including under adaptively generated proposals. We further introduce Confirm-Triggered Harmonic Spending (CTHS), which allocates risk to confirmation events rather than every proposal, improving statistical power when costly commit decisions are sparse. Across supervised learning, reinforcement learning, and optimization benchmarks, SGM filters apparent improvements caused by evaluation noise while retaining power to accept reproducible gains. In a 40-iteration ImageNet-100 self-modification trajectory, SGM produces sparse certified updates and stable incumbent performance while requiring substantially less compute than confirming every proposal. These results suggest that persistent self-improvement is naturally viewed as a trajectory-level statistical decision problem, with optimization generating candidate updates and statistical certification governing which updates are committed.

Primary area:
learning theory
Keywords:
self-modifying systems, recursive self-improvement, statistical risk control, sequential inference, irreversible updates, adaptive decision-making

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_dUGiOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper introduces Statistical Gödel Machines (SGM), a framework for adopting modifications to learning systems using statistical evidence rather than requiring a formal mathematical proof of improvement for every update. The framework begins with a candidate modification and an incumbent system, fixes the evaluation objective and candidate before confirmation, and permits adoption only when a statistically valid certificate establishes a strictly positive utility difference. Its central theoretical result is a trajectory-level guarantee: if each confirmation test has the required conditional validity and the predictable significance levels have a pathwise total no greater than a user-specified budget (\delta), then the probability of ever accepting a non-improving candidate is at most (\delta). Under the additional assumptions of a fixed utility objective and faithful state transitions, this yields a non-decreasing incumbent-utility sequence with probability at least (1-\delta). The framework does not guarantee that improvements will be discovered, that the accepted system is globally optimal, or that its future search process is better than the counterfactual process following a different decision.

The authors instantiate the framework using established statistical tools, including an anytime-valid betting-based certificate, fixed-sample testing, and an exact paired-binomial test for classification correctness. They evaluate the protocol on a controlled numerical configuration-selection problem, synthetic adaptive trajectories, and a ResNet-18 learning loop on ImageNet-100. In the synthetic experiments, SGM reduces the observed frequency of trajectories containing non-improving commitments relative to per-test confirmation, but most of that difference comes from avoiding zero-effect commitments rather than strictly harmful updates. In the numerical workload, conventional per-test confirmation has zero observed false-commit trajectories as well, with slightly higher final utility and lower evidence cost. The ImageNet-100 experiment documents one accepted checkpoint transition followed by a rejected candidate, using a fixed finite evaluation pool; it demonstrates execution of the protocol but does not estimate long-horizon error calibration or external-population generalization. The paper's principal contribution is therefore a carefully specified and auditable statistical adoption protocol, with its practical value depending on the utility-risk-cost trade-off and the validity of the evaluation assumptions.

Critical strengths
  1. A clear, correctly scoped trajectory-level safety guarantee. The paper formalizes the event that matters for persistent model modification: accepting at least one candidate whose utility does not strictly improve upon that of the incumbent. The conditional-validity requirement and summable significance-level allocation provide a straightforward guarantee without assuming independence between attempts. The paper also states the limitations of the result, including the absence of any guarantee of progress, global optimality, or improvement in counterfactual future utility. This precision makes the framework useful as a protocol for systems that make repeated adaptive changes.

  2. Excellent transparency concerning empirical limitations and statistical power. The authors distinguish zero-effect commitments from strictly harmful commitments, report verification costs, identify where conventional testing performs similarly or better, and explicitly exclude historical results whose available evidence does not support certification. The ImageNet-100 demonstration is carefully documented through checkpoint hashes, confirmation records, actual state transitions, and a held-out audit. The authors do not use that single execution trace to claim long-horizon calibration or population-level safety. This willingness to constrain claims to what the evidence supports is a substantial strength.

Critical weaknesses
  1. The incremental contribution over existing SGM work is not sufficiently differentiated.

The related-work section explicitly identifies the October 2025 preprint, SGM: A Statistical Gödel Machine for Risk-Controlled Recursive Self-Modification, as having already proposed statistical improvement certificates and a global error budget for recursive modifications. The present paper adds a more explicit treatment of conditional validity, the distinction between the adopted object and the training procedure, the binding of certificates to actual state transitions, and further controlled evaluations. These additions are useful, but the paper needs to establish more clearly which substantive contribution distinguishes this submission from the earlier SGM formulation.

The authors appropriately acknowledge that the framework does not introduce a new statistical primitive or claim universal superiority over related methods. Nevertheless, this leaves the significance question unresolved: is the central contribution a new theory of risk-controlled self-improvement, a new execution protocol, or a more rigorous empirical validation of an existing framework? These contributions can each be valuable, but their novelty and relative importance should be stated more concretely.

The related-work discussion also considers approval policies for evolving machine-learning systems, online family-wise error control, high-confidence policy improvement, and the newer SEA and Safe Harness frameworks. However, Table 10 is a comparison of scope and assumptions rather than an empirical or theoretical head-to-head study. The paper does not demonstrate that SGM's protocol provides a practically meaningful advantage over these alternatives under comparable evaluation targets and evidence budgets.

Actionable improvement: Include a precise contribution-by-contribution comparison with the cited 2025 SGM preprint, identifying which formal statements, protocol mechanisms, and experiments are new. Clarify which differences are theoretical, which are implementation-level, and which are empirical. Explain why these additions collectively constitute a substantial contribution beyond the existing statistical-testing framework.

  1. The empirical evaluation does not establish a convincing practical advantage over conventional confirmation testing.

The controlled numerical workload provides a particularly important comparison. At tolerance (10^{-3}), SGM and per-test confirmation both have zero observed false-commit trajectories among 2,000 simulations. Per-test confirmation achieves final utility of 77.68%, compared with 77.66% for SGM, while using approximately 2,634 versus 2,848 paired evaluations per trajectory. At tolerance (10^{-4}), both methods achieve the same reported final utility of 60.96%, while SGM again incurs higher evidence cost. The authors correctly acknowledge that these observations do not establish a general trajectory-level guarantee for per-test confirmation. Nevertheless, they also do not demonstrate an empirical utility-risk-cost advantage for SGM in this workload.

The synthetic experiments offer stronger evidence of a difference in trajectory-level error rates: SGM has observed family-wise error rates of approximately 0.15–0.20%, compared with 3.35–3.95% for per-test confirmation. However, the paper's post-hoc decomposition indicates that approximately 98.7–100% of the observed reduction in these settings comes from trajectories containing zero-effect errors but no harmful commitment. Per-test confirmation produces very few strictly harmful commitments in the reported simulations, while SGM has lower final utility and incurs greater verification cost. Thus, the principal empirical advantage concerns the stricter requirement to avoid even zero-improvement commitments, rather than a demonstrated reduction in consequential performance damage.

The ImageNet-100 experiment is a useful execution test but remains limited to one initialization and two confirmation attempts. The first candidate shows a large measured gain and is committed; the second candidate is rejected. The confirmation certificate concerns accuracy on a fixed finite pool sampled with replacement, not generalization to an external population. The held-out audit is descriptive and is not a test of long-horizon calibration. The authors explicitly acknowledge these limits, but the available results consequently establish executable adoption more strongly than they establish the framework's general practical value for adaptive learning systems.

Actionable improvement: Strengthen the comparison against conventional per-test testing and other appropriate error-allocation policies using a common candidate-generation process, utility definition, and total evidence budget. Report false-commit rates, strictly harmful commitments, beneficial accepted updates, retained utility, and verification cost together. For the learning-system experiment, multiple independent trajectories or a more extensive set of existing validation runs would provide substantially stronger evidence about the framework's practical behavior.

Questions for authors
  1. Novelty relative to the 2025 SGM preprint: Which specific theoretical results, algorithmic mechanisms, or guarantees in this submission are absent from the cited SGM: A Statistical Gödel Machine for Risk-Controlled Recursive Self-Modification preprint? Can the authors provide a concise comparison that isolates the substantive advances beyond the original statistical improvement certificate and global error-budget formulation? This would materially affect my assessment of the paper's originality and contribution.

  2. Practical value versus conventional confirmation: In the numerical workload, per-test confirmation and SGM both have zero observed false-commit trajectories, while per-test confirmation achieves slightly higher utility at lower evaluation cost. In the synthetic experiments, most of SGM's reduction in family-wise error comes from avoiding zero-effect commits rather than strictly harmful updates. Can the authors provide a matched-evidence-budget analysis of these policies that makes the trade-off between strictly harmful commitments, zero-effect commitments, beneficial accepts, final utility, and verification cost explicit? In particular, how often does SGM prevent a materially harmful update that the conventional comparator would actually accept?

  3. Scope of the ImageNet-100 demonstration: Are additional independently initialized learning trajectories or longer adaptive runs available for analysis? If not, can the authors clarify which conclusions about practical self-improvement are intended to follow from the two-attempt, single-initialization, finite-pool demonstration, and which remain protocol-level claims requiring future validation? This would help determine how much empirical significance should be assigned to the real-learning-system experiment.

Reviewer_QvD6OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper uses statistical tests to decide whether to accept model modifications. Candidates can be proposed based on previous results, but each acceptance must pass a separate confirmation test and be accounted for in the overall error budget. Under conditions such as test validity, the paper guarantees that the probability of accepting at least one non-improving modification throughout the entire process does not exceed a specified level. The authors demonstrate this procedure through numerical experiments, synthetic tasks, and a two-round ImageNet-100 training experiment.

Critical strengths
  1. The paper clearly distinguishes between "certifying a trained model" and "certifying a training method", and explains the conditions on which the statistical guarantees depend.

  2. The experiments report not only false acceptance rates but also missed beneficial updates and validation costs. The real-world training experiment also completes the process of continuing training after accepting a modification and then evaluating the next candidate.

Critical weaknesses
  1. SGM uses a summable error budget to control false acceptances over the long term, but the budget allocated to later tests becomes increasingly small. With a limited confirmation sample size, genuine but relatively small improvements may be difficult to accept. The paper has not yet demonstrated how many beneficial updates this procedure can retain during long-term self-improvement.

  2. The real-world learning experiment includes only two confirmation rounds, with candidates generated through simple learning rate adjustments. The current evidence supports that the statistical acceptance procedure can be executed, but there is still a considerable gap between this and long-term RSI.

  3. The real-world experiment certifies accuracy on a fixed confirmation pool (25,000 samples are drawn for each test, while the pool contains approximately 50,000 images). If the target is exactly the accuracy on the fixed confirmation pool specified in the paper, evaluating the entire pool would give the exact accuracy difference between the old and new models, without requiring a significance test. Sampling simply estimates this difference with less inference computation. If the target is accuracy on external data, repeatedly sampling from the same fixed pool does not provide the population-level generalization guarantee that the paper currently does not claim. In the current experiments, is the inference cost saved through sampling sufficient to offset the cost of potentially rejecting genuine improvements?

Questions for authors

See Weakness

Reviewer_uKhrOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper introduces Statistical Gödel Machines, a framework that replaces formal proofs with statistical certificates to authorize self-modifications in learning systems. By testing candidates against an incumbent under a controlled error budget, it guarantees that the overall risk of ever adopting a non-improving update stays below a chosen threshold. Validated on numerical benchmarks, synthetic runs, and an ImageNet-100 training loop, the method successfully prevents harmful commitments while highlighting the trade-off between strict risk control and the computational cost of statistical verification.

Critical strengths
  1. The paper replaces the formal proof of classical Gödel machines with a practical statistical test for deciding when to accept a change.
  2. The math and its assumptions are clearly stated: if the error budget is split across tests in advance, the chance of ever accepting a bad change stays under a chosen limit.
  3. The authors evaluate the framework on a numerical task, simulated runs, and an ImageNet-100 image classification loop.
  4. The paper carefully separates harmless “zero-effect” changes from truly harmful ones, and it reports how much testing each method costs.
Critical weaknesses
  1. The novelty is limited. The paper says it combines established statistical tools.
  2. The experiments don’t show that SGM is better than a simpler approach.
  3. The ImageNet-100 run is too small to support the claims.
  4. The simulated experiments use a weak test. They use a fixed betting fraction of 0.2, which the paper admits cannot reliably detect gains of 0.10 or 0.05. This makes it hard to tell whether low acceptance comes from the error budget or just from a poor test choice.
Questions for authors
  1. The error budget gets smaller with each attempt, so later improvements become hard to accept. By the 10th attempt, the chance of detecting a 0.10 gain drops from 28.7% to 4.6%. How can the system keep improving over many rounds? Would using more test data in later rounds fix this?
  2. What should the system do if the data or the way it is tested changes between attempts?
  3. Can you show a setting, such as many more rounds or many more useless proposals, where testing each change on its own leads to real problems and SGM’s error budget clearly pays off?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.