- Rating
- 3(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper studies if supervised fine-tuning causes LLM mode collapse or if diversity miscalibration can be caused by finite-sample fitting. It measures diversity using collision probablity and provides a bias-variance decomposition showing that models can become either under-diverse or over-dispersed. Also, it proves that the collision gap is bounded by the sqr root of KL divergence between the model and target distributions.
the paper evaluates the claims using synthetic languages with known ground truth distributions, human survey responses from GSS, WVS, ANES and CodeNet programming solutions. Across these, increasing the amount of SFT data generally pushes the median collision ratio toward the target level, including from both under and over diverse starting points.
Critical strengths
Strengths
-
The theoretical framing is interpretable and useful. The bias-variance-alignment decomposition is one of the important contributions of the paper. it separates the collision gap into variance and bias terms that increase collision, and an alignment term that can act in both directions. This provides a clear way of understanding why finite-sample fitting can lead to under diversity or over dispersion depending on model and target distributions.
-
The paper challenges the common focus on under diversity by showing theoretically and empirically that miscalibration can happen in both directions, depending on the model and target
-
The evaluation is broad across several settings despite missing some analysis (discussed in weaknesses). The authors evaluate four LLMs, three large survey datasets, CodeNet, and a wide range of fine-tuning dataset sizes.
-
The synthetic experiment provides a strong setting for directly testing the theoretical decomposition. Because the target probabilities are known exactly, the authors can directly estimate variance, squared bias, and alignment across repeated fits. They train 100 independent models at each of 100 sample sizes for each synthetic language, allowing the components of the decomposition to be measured directly rather than inferred indirectly.
-
The intervention is practically simple. The reported calibration effect is achieved using standard LoRA-based SFT rather than a specialized diversity-preserving objective. This makes the result practically relevant as a simple baseline
Critical weaknesses
Weaknesses
-
Collision calibration is not the same as learning the target distribution. The main claim that fine-tuning "fixes mode collapse" seems stronger than what is actually shown by . Different distributions can have the same collision probability, so a model could still assign wrong probabilities to some answers and yet look perfectly calibrated under this metric. So what is really shown is that SFT calibrates the collision statistic, which is a narrower claim than recovering the actual target distribution or its modes.
-
The survey target is not independent from the fine-tuning data. For the survey experiments, the target collision is computed using all eligible respondents, including those used in the fine-tuning and validation splits. Because of this, it is hard to tell whether the model is really learning the underlying population distribution or just getting closer to the empirical sample it was trained on. I think using a completely held-out respondent set to compute the target would make this claim much stronger.
-
The theoretical mechanism is not directly verified in the real-data experiments. Theorem 1 says that if the KL divergence between the model and target goes to zero, then the collision gap should also go to zero. But the relevant KL is not actually measured in the survey or CodeNet experiments. The paper also mentions that the survey validation loss is not the same KL used in the bound, and that the CodeNet bound is not quantitatively informative for the reported results. So the experiments show that more data moves closer to 1, but they do not directly show the full mechanism of more data lower KL better diversity calibration.
-
The CodeNet experiment does not compare human and model outputs under the same validity condition. The human target only includes accepted Python solutions, while model outputs are included as long as they parse successfully. This means incorrect but structurally different programs can still increase the measured model diversity. Also, the decoding settings are not fully matched across models, and the paper notes that the two Qwen models use different default thinking behavior. These issues make the CodeNet results harder to interpret and weaken the claim that the same calibration effect clearly generalizes to open-ended code generation.
Questions for authors
Questions for the Authors
-
The main result shows , but matching collision does not mean the model distribution approaches the target. Can the authors report a distribution-level metric (e.g. TV, JS, or KL) between model and human answer distributions for the survey experiments as increases? Does correspond to actual distributional fit, or only to matching the collision statistic?
-
The target collision is computed over all eligible respondents, including those already in the fine-tuning split. Can the authors recompute the target using only held-out respondents and report whether the trends persist? This would help with distinguishing population level generalization from fitting a sample that possibly overlaps with the training data.
-
The human target contains accepted solutions while model outputs only need to parse. Can the authors report pass rates and recompute diversity among only correct generated programs? Can they also clarify how much the unmatched Qwen decoding defaults affect the cross-model comparisons?
-
Clarification question: The paper notes that the survey validation loss is not the KL governing the collision ratio, and that the CodeNet bound is not quantitatively informative. Do the authors have measurements of the option-conditioned KL for the survey experiment? If so, does it decrease with and track ? If not, how should the experiments be interpreted as validating Theorem 1 rather than simply being consistent with it?