Model Collapse in Closed Form: Exact Gaussian Laws and Their GPT-2 Signatures

Models increasingly learn from data generated by other models, yet the mechanism of the resulting $\text{model collapse}$ is contested—mean drift or vanishing tails?—and mitigations such as data mixing are designed without an exact law to target. We answer exactly in the Gaussian testbed that both sides of the debate use as their reference model. Under recursive maximum-likelihood training, mean drift and variance contraction contribute $\text{equally}$ to the forward KL—the objective that maximum likelihood actually optimises—doubling the collapse rate; the quadratic cliff that remains is exactly half the blind spot of the plug-in variance estimator, while the reverse KL cancels it entirely, a direction asymmetry between the two divergences. Mixing real data into each generation admits an exact steady state; any schedule whose mixing fraction decays to zero diverges; and constant mixing is the best non-increasing schedule at matched real-data budget, and the only one that bounds every generation. In recursive GPT-2 training these laws appear as token-level signatures: a forward/reverse ratio near two at the first generation, sustained thereafter while both divergences saturate; extinction concentrated in a frequency band above the plug-in window; and mixing schedules that set the level of the degradation plateau—with a decaying schedule that looks safe at short horizons becoming the worst choice at triple the horizon.