acceptodds
Under review as a conference paper at ICLR 2027

Self-Training on Unprompted Samples Can Raise Utility and Lower Verbatim Extraction

Abstract

Can a language model improve from plain text sampled from itself, with no prompts, no teacher, no verifier, and no reward model? Yes, but only when the synthetic corpus is compatible with the student, a relational property of the source-student pair rather than an intrinsic property of the data. We call this the latent capability resurfacing hypothesis: weak self-training can amplify capabilities already present in the pretrained model, but only under this compatibility condition. We study the hypothesis in the minimal setting of prompt-free unconditional self-training, where base language models are fine-tuned on text generated from the BOS token alone, with no task specification or external supervision. We report three findings. First, synthetic utility is relational rather than intrinsic: self-generated data is the most effective source, same-lineage transfer outperforms stronger but differently trained sources, and cross-family transfer is substantially weaker. Second, common intrinsic proxies fail: neither benchmark-level semantic similarity nor average per-token likelihood under the student predicts which corpora help. Third, this regime produces a surprising byproduct. In controlled Pythia experiments, capability and verbatim memorization decouple: benchmark utility is preserved or improved while held-out exact-match extraction drops by over 95%, with no forget set, privacy objective, or targeted unlearning. Together, these results suggest that prompt-free self-training works by amplifying what the student already knows, not by importing structure from the data. They also reveal a regime in which capability and verbatim memorization can be separated without any explicit unlearning objective.

Primary area:
generative models
Keywords:
Synthetic Data, Privacy, Self Improvement

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_iNuyOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper asks whether a language model can get better simply by being fine-tuned on text it generated itself, with no real prompting involved. To test this, the authors build a corpus of about five million tokens by sampling from the base models at three temperatures (τ ∈ {0.75, 1.0, 1.25}), starting from nothing but a newline or end-of-text token. There's no top-k or top-p truncation, and the text is barely filtered, just an 8-gram decontamination pass against the evaluation benchmarks. Each model is then fully fine-tuned on this fixed synthetic corpus for 40 epochs. Four models at different scales: Qwen2.5-0.5B, Pythia-1B, Pythia-6.9B, and OLMo-2-13B.

Three findings stand out. First, fine-tuning on self-generated text gives measurable gains on selected reasoning, math, and coding benchmarks, and the same procedure run on Common Corpus data does not. The strongest evidence is zero-shot GSM8K: up 15.2 points for Qwen and 23.4 for OLMo. Second, the models get much worse at regurgitating their pretraining data. For the two families with public training corpora, exact-match prefix extraction drops by as much as 97% for Pythia-1B and 73% for OLMo on Wikipedia text. And this doesn't come at the cost of general performance. At certain points during fine-tuning, the models beat their original counterparts on average benchmark score while also extracting less verbatim text.

Third, whose text you train on matters. With Qwen2.5-0.5B as the student, its own generations generally work better than text from bigger or different models, including Qwen2.5-7B, Qwen3-8B-Base, and Llama-3.2-1B. The kicker is that corpora with nearly identical average negative log-likelihood under the student can still lead to very different outcomes. So how "familiar" or predictable a corpus looks to the model doesn't, on its own, tell you how useful it will be for fine-tuning.

The authors read all this as consistent with what they call latent capability resurfacing: training a model on its own outputs may bring capabilities it picked up in pretraining but doesn't normally express back to the surface. To their credit, they don't oversell this as a mechanism. The evidence stays empirical: benchmark performance tracked over 40 epochs and three training subsets, additional few-shot evaluations for Qwen and one OLMo subset, and three separate extraction criteria on OLMo. The paper shows that self-generated data can move both task performance and memorization behavior. Why it does so is left open.

Critical strengths

The setup is about as bare as you could make it, and I'd say that's the paper's main value. No prompts, no filtering, no reward, no teacher, one round, a fixed corpus. A lot of the recent "weak signal still helps" work (spurious RLVR rewards, raw self-distillation for code, power sampling) comes packaged with task prompts or verifiers, so you can't really tell where the gain came from. Here there's nowhere to hide: whatever happens has to come from the model and its own unconditional distribution. I didn't find every part of the interpretation convincing, but the testbed itself is clean and cheap to rerun, and I'd be surprised if other groups don't pick it up.

The extraction measurements are also done on models whose pretraining data is actually public, so the whole thing is checkable. The audit sets are fixed, there are three extraction criteria, and Fig. 9 tracks which base-memorized sequences make it through fine-tuning. And the authors are unusually upfront about what their numbers don't show. Near-verbatim and probabilistic extraction fall less than exact match does. Lower extraction isn't removal. And the 8-shot OLMo results in Appendix B cut against their own headline, which they say themselves. That kind of candor is rare, and honestly it's a big part of why I trust the raw numbers here even where I'd read them differently.

Critical weaknesses

My read of the utility gains is that they're changes in zero-shot response behavior, not evidence of recovered reasoning ability. The few-shot numbers make that hard to ignore. Base OLMo-2-13B gets 5.1% on zero-shot GSM8K but 72.8% with eight examples. So the zero-shot score seems to be measuring, at least in part, whether the model produces an answer that flexible-extract can parse, not whether it can solve the problem. Even the biggest reported gain, 23.4 points, only brings zero-shot accuracy to about 28%, still well below the base model's 8-shot performance. And under the 8-shot setting, the fine-tuned checkpoints actually finish 11.6–20.6 points below base (Appendix B). Qwen looks the same: gains of up to 15.2 points zero-shot shrink to at most 2.8 points with eight examples, then decline.

The authors acknowledge that the mean benchmark improvement in Fig. 1C comes mostly from zero-shot GSM8K and TruthfulQA, which means the apparent "higher utility, lower extraction" combination is pretty sensitive to the evaluation protocol. Checkpoint selection is another issue. The reported improvements are maxima over 20–40 checkpoints evaluated on the test benchmarks themselves. The 1.8-point HumanEval gain, for instance, is about three problems out of 164. I also wonder whether repeatedly training on self-generated sequences ending in an end-of-text token just teaches cleaner answers and more reliable stopping. The experiments don't rule that out, and until they do, I think claiming that self-training increases utility is premature.

My second concern is that the extraction results lack the right control. Common Corpus fine-tuning is only tested on Qwen, where extraction isn't evaluated. For Pythia and OLMo, the only comparison is against the original model. So we can't tell whether the reduction is a property of self-generated data or of fine-tuning on anything at all. Forty epochs on almost any new corpus could plausibly weaken exact 50-token greedy recall, especially given the forgetting effects in Jagielski et al. (2023). Exact-match extraction is also sensitive to small changes in generation, which fits with the smaller reductions under the less restrictive criteria.

The temperature results raise one more question. For Qwen, the \tau = 1.25 corpus has a mean NLL of 9.10 nats per token (Fig. 8). With a vocabulary of roughly 150,000, I have a hard time believing much of that text stays coherent; a good chunk of it is probably degenerate output. If OLMo produces similarly degraded samples at \tau = 1.25, the largest extraction reduction could just be noisy fine-tuning scrambling memorized sequences. That strikes me as at least as plausible as latent capability resurfacing. A matched generic-text baseline and a deliberately degenerate-text control on Pythia would go a long way toward telling the two apart.

Questions for authors

The following questions would affect my score:

  1. Could you run a matched generic-text baseline on Pythia or OLMo? Common Corpus or held-out Pile/Dolma text, same token budget, same number of optimizer steps. That would tell us whether the extraction reductions are specific to self-generated data. If generic text does the same thing, I'd chalk the result up to continued training. If it doesn't, I'd reconsider my score.
  2. What's actually in the generated text? I'd like to see per-sample NLL distributions for all three model families, the fraction of degenerate samples at each temperature, and a basic topic breakdown. How much of Qwen's output looks like math problems or exam questions, possibly echoing mid-training data? An 8-gram filter won't catch paraphrases. It would also be worth training separately on the coherent and the degenerate samples.
  3. Could the GSM8K gains be partly a formatting effect? Before and after fine-tuning, please report output lengths, end-of-sequence rates, and the fraction of answers that parse successfully. Rescoring outputs at their first natural stopping point would help separate better reasoning from cleaner answer formatting.
  4. Do the gains hold up without test-based checkpoint selection? A predetermined epoch, or a checkpoint picked on validation data or held-out NLL, would be more convincing than the best test score across 20–40 checkpoints.

A few smaller things I'd like clarified. What are the actual tokenizer-based token counts and optimizer steps for each corpus? The four-character approximation seems unreliable for degenerate multilingual output. Which TruthfulQA evaluation was used (MC1, MC2, or generation)? How much do the three training subsets overlap, given that Qwen samples are drawn with replacement and the OLMo subsets come from the same 10M-token pool? Heavy overlap would understate the variability. Finally, were Pythia extraction outputs forced to 50 tokens, as OLMo's were? If not, earlier stopping alone could reduce exact matches.

One last thing, though I wouldn't hold the score to it: a gradient-alignment or representation analysis of student-source compatibility, ideally repeated with another student model, would be genuinely interesting. I'd file that under future work.

Reviewer_j7pmOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies whether LLMs can improve by training on their own unprompted generations without any supervision. They finetuned 4 base models on 5M tokens of self-generated text, they find gains are relational: self-generated and same-lineage data outperform cross-family sources and benchmark utility improves while exact-match extraction drops >95%, with no unlearning objective.

Critical strengths
  1. The setting of the paper excludes factors like prompts verifiers rewards by construction rather than filtering them out post-hoc, making it a minimal and controlled test and the problem is rather interesting and needs a careful emperical study, so the paper's scope is rather meaningful. This is consequential because it isolates the effect of self-generated data from external supervision, allowing the study to provide clear evidence on whether and how plain self-training can work.

  2. All benchmarks are carefully decontaminated across train val and test splits. Weaker extraction criteria and negative results are reported rather than omitted. This is consequential because it reduces the risk of data leakage and selective reporting, making the main claims about capability improvement and memorization reduction more credible and reliable.

Critical weaknesses
  1. The utility gains are formatting, not capability. OLMo scores 5.1% on GSM8K zero-shot and 72.8% at 8-shot, so it can already do the task. zero-shot just does not produce a parseable answer. After training, zero-shot rises 23.4 points while 8-shot falls 20.6, with strict-match showing the same decline. Every double-digit gain in the paper is on a regex-scored generation task with a near-floor base score, and every task with a normal base score moves by single digits or goes negative. This is consequential because it suggests the headline improvements may reflect better answer formatting rather than genuine reasoning gains, weakening the central claim that self-generated data resurfaces latent capabilities.

  2. Temperature is swept only on Qwen and OLMo, learning rate only on Pythia, extraction only on Pythia and OLMo, cross-source transfer only on Qwen. The three parts of the resurfacing hypothesis therefore sit on disjoint model sets and can neither support nor refute one another. This is consequential because the paper’s main contribution is a unified hypothesis, yet the evidence is fragmented across different models; without converging results on the same model, the hypothesis remains suggestive rather than well-substantiated.

Questions for authors
  1. None of the three takeaways can be acted on. The recipe cannot be deployed, because the authors say that selecting a stopping point without tuning on the evaluation benchmarks "remains unresolved"?

  2. What do the Figure 6 curves look like for a random 5M token of the Pile and for Common Corpus, at matched lr and epochs?

Reviewer_V7saOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies one round of self-training on a fixed corpus of text generated from minimal prefixes, without task prompts, an external teacher, rewards, or quality/correctness filtering. Across four base models, it reports task-dependent utility changes and, on Pythia and OLMo, reduced extraction of documented pretraining sequences. It also shows checkpoints at which mean utility is higher and measured extraction is lower than in the base model. A source comparison suggests that transfer depends on the source–student pairing and is not predicted by mean student NLL alone. The proposed latent-capability-resurfacing explanation remains a hypothesis.

Critical strengths
  1. The paper establishes an interesting joint empirical observation. Figure 1 and Section 7 show that six of seven configurations have evaluated checkpoints with both higher mean benchmark scores and lower mean extraction counts. This is evidence at matched checkpoints, rather than a combination of unrelated utility peaks and extraction minima. For OLMo, the joint improvement is also observed in each replicate. The result is worth examining because the training uses neither task-specific supervision nor a targeted forgetting objective.
  2. The reporting makes several important boundaries visible. The paper presents training trajectories across three training sets, includes a Common Corpus utility control for Qwen, and reports both positive and negative task outcomes. The few-shot experiments and alternative extraction criteria are particularly useful: they reveal substantial protocol sensitivity and show that extraction reductions are smaller under broader criteria than under exact match. The authors also acknowledge evaluation-set peak selection and avoid claiming that lower extraction proves information removal or provides a privacy guarantee.
Critical weaknesses
  1. The utility gains are not sufficiently robust, and their reported magnitudes depend on evaluation-set selection. The largest gains are concentrated in zero-shot GSM8K. Qwen2.5-0.5B improves by up to 15.2 percentage points and OLMo-2-13B by up to 23.4, but the few-shot results substantially weaken a general capability-improvement interpretation. Qwen’s maximum 8-shot GSM8K gain is 2.8 points. For OLMo, the 8-shot evaluation ends 11.6, 12.0, and 20.6 points below the base model at epoch 40 across the three temperatures (Appendix B). That experiment uses one training set per temperature, but strict-match scoring shows the same decline. These results leave changes in prompt responsiveness as an important alternative to robust reasoning improvement. Most numerical utility gains and source rankings are selected as benchmark-specific peaks over the evaluated checkpoints. The authors disclose this, but no independently validated stopping rule establishes which model would deliver the benefits without evaluation-set selection. This does not invalidate the same-checkpoint joint effect. It does limit the practical interpretation of the reported gains, especially when different tasks improve and deteriorate at different times.
  2. The controls do not establish how specific the joint effect is to self-generated training. The Qwen Common Corpus comparison and the cross-source experiments provide relevant evidence that corpus choice matters; the paper should not be characterized as having no controls. However, the generic-text control does not cover the Pythia and OLMo experiments that establish the joint utility–extraction effect. Matched continued-training controls on those same models are needed to determine whether self-generated text has a distinctive advantage over ordinary training-induced drift or untargeted forgetting. A parameter-perturbation control, calibrated to the magnitude and distribution of the training updates, would also help test whether more generic perturbations can produce a similar joint effect. The three-training-set replication reduces concern about a single fortunate sample, but does not resolve this specificity question. Such comparisons should use the same checkpoint or hyperparameter selection rules and report both utility and extraction. I recognize that the authors present resurfacing as a hypothesis; my concern is that the remaining alternatives leave the explanatory contribution and conditions for success insufficiently established.
Questions for authors
  • Can a checkpoint chosen by an independent validation rule retain the joint improvement on held-out evaluation, with per-task zero-shot and few-shot results? What explains the reversal of the OLMo GSM8K result under 8-shot evaluation?

  • On Pythia and OLMo, how does self-generated training compare with generic-text continued training matched for actual tokens, optimization steps, and schedule? Can a calibrated parameter-perturbation control further distinguish the joint effect from generic model drift, using comparable selection budgets?

  • Does the source ranking persist after matching actual tokenizer-measured training exposure? Qwen’s nominal corpus budget is estimated from character counts, so source differences may also entail differences in actual training tokens.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.