acceptodds
Under review as a conference paper at ICLR 2027

Identifying Recursive Self-Improvement: Five Gates for Synthetic-Data Training

Abstract

A score gain over a trained baseline does not by itself identify recursive self-improvement: the comparator may have degraded, received different exposure, or been evaluated through a defective or contaminated pipeline. We formalize this distinction as a claim-conditional identification contract: every claimed effect must name its estimand, the contrast that estimates it, the factors held fixed, and the integrity checks that make that contrast interpretable. The contract yields five necessary gates — untouched-base comparison, exposure matching, verifier validation, controller activation, and decontamination — whose relevance depends on the claim being made. We apply it to DataEvolve, a protocol-frozen, three-round generator-verifier-trainer study on a sealed 2,070-item logic benchmark. The full loop exceeds the strongest declared trained baseline by 10.26 percentage points, yet remains 4.12 points below its untouched starting model: the observed gain is real, but its self-improvement interpretation is not identified. A nominal 14.55-point verifier contrast is likewise confounded; under corrected and matched conditions, verified and unverified arms score 0.6680 and 0.6681. Rebuilding a positive control with instance-disjoint data reduces it from 0.6834 to 0.6678, near the 0.6671 base. An audit-derived mixture pattern is directionally consistent across exploratory seeds, but a pre-specified fresh-holdout replication gains only 0.7 points, below its two-point minimal effect. The five-gate contract turns performance differences into auditable scientific claims and provides a concrete reporting standard for recursive training.

Primary area:
neurosymbolic & hybrid AI systems (physics-informed, logic & formal reasoning, etc.)
Keywords:
synthetic data, recursive training, self-improvement, verification, curriculum learning, training exposure, large language models, reproducibility

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 4 reviews, above 27% of the others.
Reviewer_Gq5BOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper points out that, self-improvement is not necessarily a recursive synthetic-data training pipeline that outperforms a certain training baseline, and then proposes five gates for identifying real self-improvement: untouched base, matched exposure, verifier validity, controller activation, decontamination. Then the authors apply it on DataEvolve and find out that, although DataEvolve claims to outperform a trained baseline by over 10 percent, it falls short on even outperforming a simple untouched base, and thus the self-improvement claim doesn't hold for this particular case. The authors also perform very fine-grained and careful audit on the results.

Critical strengths

(1) Timely and important topic. The AI community is suffering from various kinds of slop papers which essentially become a DDoS attack and a huge challenge to academic integrity. The paper establishes standards for identifying real claims from massive hypes in the popular subarea of RSI (recursive self-improvement) and showcases a real example of hype or faulty claims. It is meaningful to the entire AI community for restoring the order, integrity and fostering real scientific advances, and the establishment of such standards and application to existing works are timely, correct, important, and should be encouraged.

(2) Fine-grained analysis and protocol. The failure diagnosis is fine-grained and comprehensive.

Critical weaknesses

The main weakness is scope and novelty. I'm a fan of the work but I'm not sure whether or not it is appropriate as an ICLR paper. As a blogpost, I would be willing to offer my strongest support, but the amount of work and novelty remains questionable when it comes to determining whether the paper merits ICLR main conference publication. All the proposed gates, from my perspective, are common sense to real research practitioners and should be the most basic and fundamental rules that a researcher should adhere to, which has limited novelty, although I have to acknowledge that, unfortunately, it has not been the case for some works these years.

Questions for authors

I am willing to support the paper and increase my score to 4 if the authors (or other reviewers/AC) can convince me this paper is not out of scope and appropriate for main conference publication.

Q1: Could you offer some justification that the work is more suitable as a technical paper rather than a blogpost?

Q2: DataEvolve looks ambiguous in the manuscript without a proper citation or reference. Could you cite or give a pinpoint to that work (e.g. as a footnote in the paper)?

1 comment

Dear Reviewer Gq5B, thank you for your detailed feedback. We appreciate your recognition of the importance of evaluating recursive self-improvement and the value of the failure analysis. We have clarified the technical contribution and scope of the case study and plan to incorporate these changes into the revised manuscript.


  1. W1 & Q1. The technical contribution is not any individual check, but a reusable design that distinguishes four questions: whether training improves on the starting model and whether loop-generated data, verification, or adaptive sampling add value. In the same experiment, the loop’s mean score differed by +10.26 percentage points relative to the strongest pre-specified fine-tuned baseline, but by −4.12 relative to the untouched starting model. These contrasts answer different questions. In §4, we also use a new holdout and a pre-specified protocol to test a hypothesis about mixing data that emerged from our diagnosis: the mixture arm scored 0.71 percentage points above the starting model, below the 2-point target. This design lets readers verify which conclusions the same result actually supports.
  2. Q2. DataEvolve is the three-round data pipeline defined in §3.3 of this paper, not a separate paper. A frozen Qwen3-8B checkpoint solves the tasks; fine-tuned models do not become solvers in later rounds. We plan to add a footnote at the first mention pointing to §3.3 and to the task generator, answer checker, controller, and training and evaluation scripts in the supplementary material.

Thank you again for your review.

Reviewer_eaEYOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper asks what a reported gain from a self-training loop (the model generates problems, a checker filters its answers, and the model trains on whatever passes) has actually shown. The authors point is that "beats a trained baseline" often gets read as "the model improved itself", even though the baseline may have degraded, seen less data, or been evaluated on a leaky pipeline. To fix this, they propose a "claim-conditional identification contract". Each claim (the model got better, recursive data helped, the verifier helped, the controller helped, it generalizes) has to state what it measures, which comparison measures it, what is held fixed, and which checks must pass. In practice this comes down to five gates: compare against the untouched base model, match training exposure, validate the verifier, show the controller actually did something, and decontaminate every training arm.

The evidence is mainly a case study of the authors own pipeline, DataEvolve: Qwen3-8B with LoRA, three rounds, three seeds, and a sealed 2,070-item logic benchmark. In the pre-specified run, the loop beats the best trained baseline by about 10 points but ends up about 4 points below the untouched base model. A post-hoc audit then turned up a checker bug, truncated training targets, a controller that never moved, unequal exposure across arms, and a "strong" control that had seen test prompts. Once these are repaired (post hoc, so exploratory), all arms land around the base model's score. The verifier adds nothing when almost every answer is already correct, though small exploratory runs suggest it helps when errors are common. The authors also try an idea that came out of the audit: mixing loop data with clean outside data. It looks like +1.8 points at first, but on a fresh pre-registered holdout it gains only +0.7, below their own 2-point threshold. Finally, they check three published systems (LMSI, Self-Rewarding, SPIN) against the gates and find that each leaves at least one gate open or unreported.

Overall, the paper presents itself as a methodology and reporting-standard contribution rather than a new training method. Its main evidence is that applying the gates to one experiment reverses that experiment's headline conclusion.

Critical strengths
  1. The paper demonstrates its central concern within a single experiment rather than only arguing it in the abstract. The pre-specified v1 run exceeds the best trained baseline by about 10 points, yet falls about 4 points below the untouched base model (Table 3). Comparisons of this kind are frequently reported as evidence of self-improvement, so seeing the conclusion reverse once the base model is included is an effective illustration. I note that, with only three seeds, neither contrast is statistically resolved (Table 15). However, all three v1 seeds fall below the base model (A.8), so the direction is consistent.

  2. Careful separation of planned and post-hoc evidence. The authors retain the frozen v1 endpoint, label every post-audit repair as exploratory, and test their own audit-derived mixture hypothesis on a fresh, pre-specified holdout. When the result (+0.7) fell below their own two-point threshold, they reported it as a failed confirmation. Negative results of this kind are rarely published, and this one shows that the proposed contract can restrain authors from over claiming their own findings.

Critical weaknesses
  1. The case study does not test recursive self-improvement, the claim the paper is built around.

  2. The headline results are stated with more certainty than the paper's own statistics support. Table 15 labels both main v1 contrasts "Exploratory", with 95% intervals that include zero ([-0.073, +0.278] and [-0.131, +0.049]), yet Table 3 calls them "confirmatory" and the abstract says "the observed gain is real".

  3. The repaired experiments, which carry the positive conclusions, have problems the paper's own standards would flag.

  • Exposure: the abstract, Section 4.3 and Table 8 describe v3b as matched, but Table 12 shows 1,941-1,947 pairs and 244 updates for "Loop, last round only" against 1,400 pairs and 176 updates for "Static, matched" (about 39% more).
  • Provenance: the static seed-2 path in v3b_aggregate_20260917.json is not in the supplement. The only seed-2 folder, v3_base1c_s2_re_s41001, has the same acc_overall, acc_L1 and acc_L2 as seed 3 (0.6691 / 0.6586 / 0.6745), although the adapters differ.
  • Holdout: in hold2_aggregate_20260924.json, generic_vs_base is 0.8727 pp, above the pre-registered MIX0.35 arm (0.709 pp), and A.19 does not report that the generic-only mean exceeds the mixture's.
  • Verifier contrast (A.17, 69.5% acceptance): the arm sizes are not reported.
  1. The framework is largely established practice under new names. The five gates are standard experimental hygiene, and the claim-to-check mapping the paper presents as its contribution (Sections 2 and 5) is what careful ablation design already involves. No identification result is derived, and the word "assumption" appears nowhere in the paper.
Questions for authors

Q1 Is the loop recursive? In exp_main.py and exp_v2.py, each round's data is generated by a solver built on model_dir, and each round trains a fresh adapter that is never reused. The controller receives "delta": 0.0 (exp_main.py:69, exp_v2.py:249), and in controller.py the R-div rule never fires. In what sense is this recursive self-improvement?

Q2 What exactly did the v1 Knights checker do, and how many items were wrongly accepted and wrongly rejected? The comment at backends_full.py:218-219 says "any non-empty response vacuously passed", but roles(resp) == roles(answer) with an empty gold map would pass only unlabeled responses and reject labeled ones. Is the 3/5,843 failure rate in Section 4.3 computed over closed traces only (exp_v2.py:222), and what was the rate before the closure filter?

Q3 Is the loop training data disjoint from the 700 L1 test items? sample_by_omega applies no exclusion, and neither exp_main.py nor exp_v2.py calls decontam.py. Please report exact and near-duplicate overlap per arm, as G5 requires.

Q4 Is v3b exposure-matched? Table 12 shows 1,941-1,947 pairs and 244 updates for the loop versus 1,400 and 176 for static. Also, the static seed-2 entry in v3b_aggregate_20260917.json points to a folder (v3_base1c_s2_sRUN-v3b_base1c_s2_t3) that is not in the supplement; only v3_base1c_s2_re_s41001 is present. Its acc_overall, acc_L1 and acc_L2 (0.6691 / 0.6586 / 0.6745) equal both that aggregate entry and seed 3, although the adapters differ. Why was seed 2 re-run under a different folder name, and are the seed-2 and seed-3 training sets different?

1 comment

Dear Reviewer eaEY, thank you for reading the paper and supplement so carefully. We appreciate your highlighting the reversal between the two baseline comparisons and the distinction between pre-specified and exploratory analyses. In response to your feedback, we have narrowed our conclusions and clarified the experimental setup, and we will incorporate these changes in the revised manuscript.


  1. W1 & Q1 (Multi-round generation). Each round uses the same frozen Qwen3-8B checkpoint to solve the problems, then trains a fresh adapter. The adapter is not reused to solve problems in the next round, and its performance is not fed back to the controller. The sampling ranges stayed fixed in the original experiment. We now frame the case study as a comparative study of multi-round data generation, not an empirical test of recursive model self-improvement.
  2. W2 & W4 (Evidence and contribution). We pre-specified the comparisons, not the conclusions. The exploratory 95% intervals for both comparisons cross zero; all three seeds scored below the starting model, but this is a descriptive result. Our contribution is to match claims to the comparisons needed to support them and show how two such comparisons change the interpretation of the same experiment.
  3. W3 & Q4 (Budgets, records, and holdout evaluation). The final round of the v3b loop arm had 1,941–1,947 training pairs and 244 updates. The two capped control arms each had 1,400 pairs and 176 updates, so the budgets were not matched. The original evaluation file for seed 2 in the static arm was not included in the supplement, but the retained re-evaluation matches the aggregate report on both overall and per-layer scores. Seeds 2 and 3 have identical scores on these metrics, but their adapter records and recorded counts of distinct prompts differ (686 and 634, respectively). The revision will distinguish the original evaluation records, the re-evaluation, and the sources of the training data. On the new holdout, the pre-specified mixture arm scored 0.71 percentage points above the starting model, below both the pre-specified target of 2 percentage points and the generic-only arm's gain of 0.87 percentage points.
  4. W3 & Q2 (Verifier). Your reading of the code is correct. A case mismatch caused the reference role map to parse as empty, so the verifier accepted incorrect answers with no parseable roles and rejected answers whose correct roles were parsed. We removed the previously reported 3/5,843 figure; the revision will explain how false accepts and false rejects should be counted before and after filtering. The 69.5% figure is the pass rate in a pilot with a 1,024-token solving budget. The revision will distinguish it from the actual training sample counts and training budgets in each arm and will not present the comparison as budget-matched.
  5. Q3 (Problem overlap). The original pipeline did not check for overlap round by round. For one archived seed, the static and random synthetic training pools contained 253 and 245 unique problem prompts that exactly matched L1 test prompts, respectively. These counts capture only exact duplicates in the specified control arms and do not support a claim that the loop arm did not overlap with L1. The revision will clarify this scope and add a plan to check for exact and near duplicates by arm and seed.

Thank you again for your careful review and suggestions.

Reviewer_8N1oOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper argues that beating a trained baseline does not prove a model improved itself. It proposes an “identification contract”: every claim should state what it measures, which comparison measures it, what is held fixed, and which checks were done. From this it derives five “gates”:

  1. Compare against the untouched starting model.
  2. Match training exposure (data, tokens, updates).
  3. Check that the verifier works.
  4. Check that the adaptive controller actually changes its behavior.
  5. Remove test-set overlap from training data.

The authors apply this to their own three-round generate–verify–train loop (Qwen3-8B, LoRA, logic tasks). The loop beats the best trained baseline by 10.26 points but is 4.12 points below the untouched model. An audit then finds several bugs: a broken checker, truncated training targets, a controller that never moved, unequal exposure, and test overlap in the positive control.

Critical strengths
  1. The main message is correct and useful. Self-improvement claims should be compared against the starting model, with matched exposure and clean evaluation.
  2. The paper is honest. It reports its own failed experiment, labels post-hoc results as exploratory, and runs a pre-registered replication that does not confirm its own hypothesis.
  3. The low-solve verifier experiment is a nice idea. It shows that verification helps only when there are errors to filter.
Critical weaknesses
  1. Honestly, I am slightly doubtful about the contributions. The five gates are standard experimental practice, not a very new contribution. Comparing to the starting model, controlling for compute and data, checking that a measurement tool works, confirming the treatment actually happened, and avoiding test leakage are long-standing advice in ML (e.g., Dodge et al., “Show Your Work”, EMNLP 2019; Pineau et al., JMLR 2021; Bouthillier et al., MLSys 2021).

  2. The causal vocabulary (“estimand”, “identification”) is used loosely: there is no formal identification result, no stated assumptions, and the “identification graph” (Figure 2) is not analyzed as a causal model. The gates are also called “necessary” without proof.

  3. The case study mainly shows that a buggy experiment gives wrong answers. The “identification reversal” comes from the authors’ own implementation errors: a checker bug, targets truncated at 256 tokens, a controller that never moved, and mismatched exposure. Apply the gates to at least one published system by re-running it with and without the missing controls, and show that a published conclusion changes.

  4. The untouched base is a single greedy evaluation with no uncertainty, yet many conclusions depend on differences of 0.01–0.1 points from it (for example 0.6680 vs. 0.6671). Greedy decoding is not deterministic across runs, and the base model’s own run-to-run noise is unknown.

  5. The writing makes simple points hard to read. Terms like “claim-conditional identification contract”, “gates”, “evidence status” and “admissible conclusions” wrap ordinary advice in heavy language.

Questions for authors

Please address the weaknesses.

1 comment

Dear Reviewer 8N1o, thank you for your thoughtful feedback. We appreciate your recognition of the starting-model comparison, the transparent reporting of negative results, and the verifier experiment in the low-acceptance setting. We have clarified the comparisons and simplified the terminology, and plan to incorporate these changes into the revised manuscript.


  1. W1. The five checks draw on standard experimental practice. Section 3.1 distinguishes four questions: whether training improves on the starting model, and whether loop-generated data, verification, and adaptive sampling each add value. We specify the control needed for each.
  2. W2 & W5. We now use plain terms such as “comparisons” and “checks” and state the conditions for the comparisons in §3.1. Figure 2 is a schematic; it is not used for formal causal identification.
  3. W3. The observed difference changes sign when the same result is compared with the starting model. Checking for implementation issues is a separate analysis. Section 5 analyzes published results rather than rerunning the systems, and we will make this distinction explicit. For instance, a SPIN replication could retain the original baseline and add a static control matched on training examples, tokens, and update steps to see whether the two comparisons lead to the same conclusion.
  4. W4. The starting-model score of 0.6671 comes from a single evaluation with greedy decoding. The difference from 0.6680 is descriptive and does not establish equivalence. The revision will make clear that a single evaluation does not estimate run-to-run variability and will explain how to report that variability separately.

Thank you again for your careful review.

Reviewer_g13gOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes a claim-conditional identification contract for recursive synthetic-data training. It distinguishes improvement over the starting model from contributions attributable to recursive data, verification, or adaptive control, and organizes the required controls into five gates: untouched-base comparison, exposure matching, verifier validation, controller activation, and decontamination.

The framework is illustrated through DATAEVOLVE, a three-round Qwen3-8B training study evaluated on 2,070 logic-reasoning items with three training seeds. The full loop exceeds the strongest declared trained baseline by 10.26 percentage points but falls 4.12 points below the untouched model. Subsequent audits and exploratory repairs examine implementation defects, exposure differences, and contamination. A separate 1,100-item holdout with five seeds tests an audit-derived mixture hypothesis, yielding a 0.71-point improvement that does not meet the pre-specified two-point target. The principal contribution is a framework for interpreting experimental claims, supported by an extensively documented case study and desk audits of related systems.

Critical strengths
  1. A useful connection between scientific claims and experimental controls. The paper explicitly distinguishes absolute training improvement, recursive-data value, verifier value, and controller value. This helps identify why an ablation may fail to estimate the effect named in a paper’s conclusions. The observed reversal between the trained-baseline comparison and the untouched-base comparison makes this distinction particularly concrete.

  2. Substantial disclosure of unsuccessful and exploratory results. The authors report implementation failures, contamination, uncertainty across training seeds, and the failure of the fresh-holdout experiment to meet its practical-effect target. They also acknowledge that coupled repairs cannot identify individual defect contributions and that similar means do not establish equivalence. These disclosures make the case study useful for understanding experimental failure modes.

Critical weaknesses
  1. The principal repaired comparison does not demonstrably satisfy the paper’s own exposure-matching gate. Section 4.3 and Tables 8–9 describe v3b as exposure matched. However, Table 12 reports 1,941–1,947 training pairs, 1.65–1.69M tokens, and 244 updates for the loop, versus 1,400 pairs, approximately 1.14–1.22M tokens, and 176 updates for the static/random controls. These are material differences under G2, which explicitly requires matching examples, tokens, and optimization exposure. The full-pool experiments have closer budgets, but the manuscript must explain their relationship to the recursive estimand rather than treating them as validation of the v3b matching claim. This matters because the methodological contribution depends on demonstrating that the audit can produce interpretable repaired comparisons.

  2. Inconsistent descriptions of the realized interventions undermine the central attribution audit. Section 4.2 describes the checker defect as rejecting valid responses, whereas Appendix A.3 and Table 10 describe vacuous acceptance of malformed responses. Section 4.2 also describes stored targets as extracted short answers, while Appendix A.4 states that v1 stored full responses subject to truncation. These mechanisms could coexist across conditions or versions, but the manuscript does not provide that mapping. Since the contribution rests on diagnosing what actually happened, readers need an unambiguous account linking each defect to the affected data, runs, and conclusions.

Questions for authors
  1. How should the v3b “matched exposure” claims be reconciled with Table 12? Please identify which reported comparison satisfies G2 and specify whether the budgets cover only final-model fitting or the complete recursive procedure. Would correcting the matching description change any headline conclusions?

  2. Did the Knights defect cause false rejection, false acceptance, or both under different execution paths? Similarly, which v1 conditions used extracted answers versus full, potentially truncated responses? Can you substantiate the mapping with version-specific code, representative records, and affected-example counts?

1 comment

Dear Reviewer g13g, thank you for taking the time to give us detailed feedback. We appreciate your recognition of how we distinguish the controls needed for different claims and report results that fell short of expectations. We have checked both discrepancies you flagged and updated the relevant wording, and will include those clarifications in the revised manuscript.


  1. W1 & Q1 (Training budget). You are right: v3b used the same response format and closure screen, but did not meet G2. We have corrected this in §4.3. For the final fine-tuning run, the loop arm used 1,941–1,947 training pairs, 1.65–1.69 million tokens, and 244 updates. Each control used 1,400 pairs, with 1.14–1.22 million tokens and 176 updates. Using the full pools brings the budgets closer only for the final fine-tuning run: it compares training on pooled data, not the full multi-round procedure. The observed differences in the original study are unchanged (+10.26 percentage points relative to the stronger of the pre-specified trained baselines and −4.12 relative to the starting model), but they cannot be attributed to individual components.
  2. W2 & Q2 (Verifier and training targets). Both errors came from the same Knights checker: it lowercased the model response but not the reference answer, so the parser returned an empty role map for the reference. An incorrect response with no parseable roles could pass, while the checker rejected responses from which it parsed the correct roles. The main v1 loop stored generated responses subject to a 256-token generation cap, not extracted short answers. Short reference answers were used only in a separate diagnostic. Both failure modes can be verified from the original code, but the retained records do not provide corpus-wide counts. In the revision, we will map code versions to experimental arms, illustrate both errors, and distinguish the generation cap from truncation at training time.

Thank you again for your careful review.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.