- Rating
- 3
- Confidence
- 3(+1)
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper introduces MIXSD, a self-distillation method for injecting new knowledge into language models while reducing catastrophic forgetting. MIXSD constructs training sequences by mixing tokens from an expert conditional that observes the target knowledge and a naive conditional that reflects the base model’s original behavior, followed by standard NLL training on the verified mixed outputs. Experiments on synthetic factual and arithmetic tasks, SimpleQA, and MQuAKE show that MIXSD generally achieves a better knowledge-acquisition and capability-retention trade-off than SFT, OPSD, LoRA, and several knowledge-editing baselines across multiple model scales.
Critical strengths
-
MIXSD provides a simple and intuitive way to inject new knowledge while keeping the supervision closer to the base model’s original generation distribution.
-
The experiments are comprehensive, covering factual recall, arithmetic function learning, open-domain knowledge, knowledge editing, multiple model scales, and several relevant baselines and ablations.
-
The paper provides useful analyses beyond aggregate accuracy, including target likelihood, parameter-update directions, error patterns, training-seed variation, and the acquisition-retention trade-off.
Critical weaknesses
i) The experiments do not clearly isolate the effect of mixed token-level sampling from the other changes introduced during data construction.
MIXSD replaces the short answers used by standard fine-tuning with substantially longer reasoning sequences generated by the base model. The resulting training data therefore differ not only in how tokens are sampled, but also in sequence length, style, total training tokens, and optimization steps. More importantly, the =0 variant already provides much of the reported retention improvement in several settings, even though it uses only the expert-conditioned distribution and performs no mixing with the naive distribution. This makes it difficult to determine how much of the gain actually comes from the proposed mixed contextual distillation.
Prior work has shown that training on self-generated data can itself preserve a language model’s existing capabilities [a]. The paper should therefore include controlled experiments that match sequence length, training-token count, and optimization steps across variants. Comparisons between mixed sequences and equally long sequences generated entirely from the expert or base-model distribution would help establish whether token-level mixing provides an additional benefit.
ii) The method relies heavily on verification and repeated generation, but the contribution of this selection process is not separated from the proposed distillation mechanism.
MIXSD retains a generated sequence only when it passes a task-specific correctness check, with up to 100 generation attempts for each example. Table 6 reports high retry rates in several settings, while the paper does not report the average number of attempts or generated tokens required for each accepted sequence. This filtering process changes the training distribution by keeping responses that the base model can already express correctly, and it may account for part of the improvement attributed to MIXSD. Related work has shown that selecting or regenerating self-produced rationales according to answer correctness can substantially improve downstream training [b].
The dependence on automatic verification also limits the current evidence for broader knowledge injection. Exact checks are relatively straightforward for the factual and functional tasks considered here, but many realistic updates involve long-form, ambiguous, or open-ended knowledge for which correctness cannot be determined automatically. The paper should separate the effects of mixing and verification through an ablation without rejection filtering, report the full generation cost, and explain how the method would operate when a reliable verifier is unavailable.
[a] Mitigating Catastrophic Forgetting in Large Language Models with Self-Synthesized Rehearsal. ACL 2024.
[b] STaR: Bootstrapping Reasoning With Reasoning. NeurIPS 2022.
Questions for authors
-
When sequence length, total training tokens, and optimization steps are matched, does mixed token-level sampling still outperform sequences generated entirely from the expert-conditioned or base-model distribution?
-
How much of the improvement comes from verification and repeated sampling rather than contextual mixing, and how would MIXSD be applied when no reliable automatic verifier is available?
Thank you for the detailed responses and additional experiments. The new context-distillation controls provide stronger evidence for the benefit of mixing, and the reported retry counts and generation costs are helpful. These additions clarify several of my concerns and strengthen my confidence in the empirical results. Exact training-token matching and reliance on verification remain limitations. I maintain my original rating of 3 (weak acceptance).