- Rating
- 2
- Confidence
- 3
- Informativeness
- 2(−1)
- Presentation
- 3
- Soundness
- 2(−1)
Read review
Paper summary
This paper investigates the rapid loss of lexical diversity in language models during recursive self-training. While theoretical analyses attribute "model collapse" to the gradual accumulation of statistical errors, empirical studies often reveal severe phrase repetition occurring within just a few generations. The authors discover that this rapid degradation stems primarily from the characteristics of inference samplers (such as those used in vLLM), rather than from flaws within the models themselves. When requests within a batch share a fixed random seed—and this seed is reused across generations—the sampler repeatedly injects the same sampled noise. This creates a feedback loop, or "resonance," which continuously reinforces specific tokens and encodes them into the model's weights. Crucially, decoupling this noise by assigning an independent random seed to each request entirely eliminates this rapid collapse, restores lexical diversity, and brings empirical results into alignment with theoretical predictions.
Critical strengths
- This paper reveals an interesting and potentially overlooked interaction between sampling at inference time and recursive training. This finding underscores the importance of implementation details when interpreting experiments on model collapse.
- The factorial experimental design—which separates sharing across requests from replay across generations—is well-justified. Cross-decoding experiments also provide useful evidence for distinguishing between repetition induced by sampling and changes encoded within the model weights.
- The investigation into token-level sampling behavior, along with the reconstruction of seed-dependent sampling patterns, offers compelling mechanistic evidence for the observed phenomena.
Critical weaknesses
- The primary finding concerns implementation-related sampling biases within vLLM. While the interplay between stochasticity and recursive training is of empirical interest, the underlying mechanisms are largely consistent with well-established principles of correlated sampling and feedback amplification. The proposed solution restores standard statistical assumptions rather than introducing novel learning principles or algorithmic contributions.
- The paper assesses "model collapse" primarily through metrics such as lexical diversity, self-BLEU scores, and the perplexity of generated samples. While these metrics reflect output distributions, they do not directly demonstrate a degradation in the underlying model’s predictive capability or its performance on downstream tasks. Cross-decoding results further suggest that a significant portion of the observed collapse can be attributed to the sampling process during inference. However, in the absence of held-out language modeling evaluations or downstream task benchmarks, it remains unclear whether this phenomenon truly constitutes "model collapse."
- The experiments rely on relatively small synthetic datasets (2,100 examples per round), a single fine-tuning regime, and a setup where synthetic data entirely replaces—rather than supplements—real data. These choices may amplify the impact of repetitive or correlated samples. The paper does not investigate how this phenomenon scales with respect to dataset size, training intensity, or the ratio of real to synthetic data; thus, its relevance to modern, large-scale post-training pipelines remains uncertain.
- Most experimental conditions are evaluated based on a single training chain, with limited replication across different random seeds. This is particularly concerning, given that the proposed mechanism explicitly depends on the specific characteristics of individual stochastic realizations. Consequently, more extensive multi-seed experiments are required to characterize the variability, generality, and robustness of the reported "resonance" effects.
- The theoretical analysis is based on a phenomenological scalar update that is not derived from the actual neural network optimization process. It overlooks prompt dependency, autoregressive token interactions, and parameter sharing, rendering it incapable of predicting relative sensitivities across different checkpoints. As such, while the analysis offers an intuitive explanation, it provides limited predictive power or theoretical insight into the observed training dynamics.
Questions for authors
- Does the same phenomenon occur in other inference engines (e.g., Transformers or SGLang) when comparable sampling dependencies are introduced? How frequently does this type of configuration arise in practical synthetic data generation pipelines?
- How does this effect vary with the size of the synthetic dataset, the number of training epochs, and the proportion of real data included in the mix? Specifically, does the rapid performance decay persist when training on significantly larger synthetic datasets?
- Can deduplication or frequency-based reweighting mitigate the observed effect without altering the sampling configuration? This would help clarify the relative contributions of repeated training examples versus sampling feedback loops across generations.
- Does the reuse of seed data degrade language modeling performance or accuracy on downstream tasks? Can the authors distinguish between a decline in model capability and the repetition-related issues primarily driven by sampling during inference?