- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The study says defenses against copying a model through outputs should be evaluated after copying model receives further reinforcement learning, rather than immediately after training on the teacher's answers. It first compares training on teacher outputs, reinforcement learning, and their combination across several small language models on mathematics tasks. It shows that low and moderate output poisoning can lose much of its effect after reinforcement learning in a Qwen2.5-0.5B experiment. Experiments include multiple teacher providers, controls using the expander without summaries, and additional training rounds. The main contribution is an important correction to how distillation defenses are evaluated.
Critical strengths
- Main question is clearly testable.
- The study includes training without teacher data.
- The poisoning study varies defense strength. Comparing several levels exposes the trade-off between harming the copying model and degrading the teacher's useful answers.
- The closed-model study adds practical relevance.
- Additional training rounds probe a possible performance ceiling.
- Provides substantial experimental detail.
Critical weaknesses
- The combined distillation-and-reinforcement-learning experiments use students with at most 3 billion parameters and mathematics tasks.
- The training methods aren't compared under a common total budget. Sharing questions does not match teacher-query costs, generated tokens, or training work.
- Roles of summaries and final answers aren't fully separated.The closed-model attack uses both. An answer-only expansion control is needed to establish how much extra useful information the summaries provide.
- Trace authenticity is checked mainly through length. Matching reported token counts is useful, but different reasoning texts can have matching lengths. Stronger validation would help establish that the extracted full-trace reference is authentic.
- The claim that the attack is cheap needs measurement.
- The general claim about output-level defenses is too strong.Failure of the tested poisoning and summarization settings does not show that every response-level defense is ineffective or economically pointless.
Questions for authors
- Why does poisoning lose its effect in the Qwen experiment but retain a substantial effect in the Llama experiment?
- Could a defense be useful by raising the attacker's cost, even when sufficient training eventually recovers accuracy?
- Which results would be needed before extending the conclusions from small mathematics models to large coding or tool-using models?