- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper asks whether a language model can get better simply by being fine-tuned on text it generated itself, with no real prompting involved. To test this, the authors build a corpus of about five million tokens by sampling from the base models at three temperatures (τ ∈ {0.75, 1.0, 1.25}), starting from nothing but a newline or end-of-text token. There's no top-k or top-p truncation, and the text is barely filtered, just an 8-gram decontamination pass against the evaluation benchmarks. Each model is then fully fine-tuned on this fixed synthetic corpus for 40 epochs. Four models at different scales: Qwen2.5-0.5B, Pythia-1B, Pythia-6.9B, and OLMo-2-13B.
Three findings stand out. First, fine-tuning on self-generated text gives measurable gains on selected reasoning, math, and coding benchmarks, and the same procedure run on Common Corpus data does not. The strongest evidence is zero-shot GSM8K: up 15.2 points for Qwen and 23.4 for OLMo. Second, the models get much worse at regurgitating their pretraining data. For the two families with public training corpora, exact-match prefix extraction drops by as much as 97% for Pythia-1B and 73% for OLMo on Wikipedia text. And this doesn't come at the cost of general performance. At certain points during fine-tuning, the models beat their original counterparts on average benchmark score while also extracting less verbatim text.
Third, whose text you train on matters. With Qwen2.5-0.5B as the student, its own generations generally work better than text from bigger or different models, including Qwen2.5-7B, Qwen3-8B-Base, and Llama-3.2-1B. The kicker is that corpora with nearly identical average negative log-likelihood under the student can still lead to very different outcomes. So how "familiar" or predictable a corpus looks to the model doesn't, on its own, tell you how useful it will be for fine-tuning.
The authors read all this as consistent with what they call latent capability resurfacing: training a model on its own outputs may bring capabilities it picked up in pretraining but doesn't normally express back to the surface. To their credit, they don't oversell this as a mechanism. The evidence stays empirical: benchmark performance tracked over 40 epochs and three training subsets, additional few-shot evaluations for Qwen and one OLMo subset, and three separate extraction criteria on OLMo. The paper shows that self-generated data can move both task performance and memorization behavior. Why it does so is left open.
Critical strengths
The setup is about as bare as you could make it, and I'd say that's the paper's main value. No prompts, no filtering, no reward, no teacher, one round, a fixed corpus. A lot of the recent "weak signal still helps" work (spurious RLVR rewards, raw self-distillation for code, power sampling) comes packaged with task prompts or verifiers, so you can't really tell where the gain came from. Here there's nowhere to hide: whatever happens has to come from the model and its own unconditional distribution. I didn't find every part of the interpretation convincing, but the testbed itself is clean and cheap to rerun, and I'd be surprised if other groups don't pick it up.
The extraction measurements are also done on models whose pretraining data is actually public, so the whole thing is checkable. The audit sets are fixed, there are three extraction criteria, and Fig. 9 tracks which base-memorized sequences make it through fine-tuning. And the authors are unusually upfront about what their numbers don't show. Near-verbatim and probabilistic extraction fall less than exact match does. Lower extraction isn't removal. And the 8-shot OLMo results in Appendix B cut against their own headline, which they say themselves. That kind of candor is rare, and honestly it's a big part of why I trust the raw numbers here even where I'd read them differently.
Critical weaknesses
My read of the utility gains is that they're changes in zero-shot response behavior, not evidence of recovered reasoning ability. The few-shot numbers make that hard to ignore. Base OLMo-2-13B gets 5.1% on zero-shot GSM8K but 72.8% with eight examples. So the zero-shot score seems to be measuring, at least in part, whether the model produces an answer that flexible-extract can parse, not whether it can solve the problem. Even the biggest reported gain, 23.4 points, only brings zero-shot accuracy to about 28%, still well below the base model's 8-shot performance. And under the 8-shot setting, the fine-tuned checkpoints actually finish 11.6–20.6 points below base (Appendix B). Qwen looks the same: gains of up to 15.2 points zero-shot shrink to at most 2.8 points with eight examples, then decline.
The authors acknowledge that the mean benchmark improvement in Fig. 1C comes mostly from zero-shot GSM8K and TruthfulQA, which means the apparent "higher utility, lower extraction" combination is pretty sensitive to the evaluation protocol. Checkpoint selection is another issue. The reported improvements are maxima over 20–40 checkpoints evaluated on the test benchmarks themselves. The 1.8-point HumanEval gain, for instance, is about three problems out of 164. I also wonder whether repeatedly training on self-generated sequences ending in an end-of-text token just teaches cleaner answers and more reliable stopping. The experiments don't rule that out, and until they do, I think claiming that self-training increases utility is premature.
My second concern is that the extraction results lack the right control. Common Corpus fine-tuning is only tested on Qwen, where extraction isn't evaluated. For Pythia and OLMo, the only comparison is against the original model. So we can't tell whether the reduction is a property of self-generated data or of fine-tuning on anything at all. Forty epochs on almost any new corpus could plausibly weaken exact 50-token greedy recall, especially given the forgetting effects in Jagielski et al. (2023). Exact-match extraction is also sensitive to small changes in generation, which fits with the smaller reductions under the less restrictive criteria.
The temperature results raise one more question. For Qwen, the \tau = 1.25 corpus has a mean NLL of 9.10 nats per token (Fig. 8). With a vocabulary of roughly 150,000, I have a hard time believing much of that text stays coherent; a good chunk of it is probably degenerate output. If OLMo produces similarly degraded samples at \tau = 1.25, the largest extraction reduction could just be noisy fine-tuning scrambling memorized sequences. That strikes me as at least as plausible as latent capability resurfacing. A matched generic-text baseline and a deliberately degenerate-text control on Pythia would go a long way toward telling the two apart.
Questions for authors
The following questions would affect my score:
- Could you run a matched generic-text baseline on Pythia or OLMo? Common Corpus or held-out Pile/Dolma text, same token budget, same number of optimizer steps. That would tell us whether the extraction reductions are specific to self-generated data. If generic text does the same thing, I'd chalk the result up to continued training. If it doesn't, I'd reconsider my score.
- What's actually in the generated text? I'd like to see per-sample NLL distributions for all three model families, the fraction of degenerate samples at each temperature, and a basic topic breakdown. How much of Qwen's output looks like math problems or exam questions, possibly echoing mid-training data? An 8-gram filter won't catch paraphrases. It would also be worth training separately on the coherent and the degenerate samples.
- Could the GSM8K gains be partly a formatting effect? Before and after fine-tuning, please report output lengths, end-of-sequence rates, and the fraction of answers that parse successfully. Rescoring outputs at their first natural stopping point would help separate better reasoning from cleaner answer formatting.
- Do the gains hold up without test-based checkpoint selection? A predetermined epoch, or a checkpoint picked on validation data or held-out NLL, would be more convincing than the best test score across 20–40 checkpoints.
A few smaller things I'd like clarified. What are the actual tokenizer-based token counts and optimizer steps for each corpus? The four-character approximation seems unreliable for degenerate multilingual output. Which TruthfulQA evaluation was used (MC1, MC2, or generation)? How much do the three training subsets overlap, given that Qwen samples are drawn with replacement and the OLMo subsets come from the same 10M-token pool? Heavy overlap would understate the variability. Finally, were Pythia extraction outputs forced to 50 tokens, as OLMo's were? If not, earlier stopping alone could reduce exact matches.
One last thing, though I wouldn't hold the score to it: a gradient-alignment or representation analysis of student-source compatibility, ideally repeated with another student model, would be genuinely interesting. I'd file that under future work.