- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes SubShare, which decides which layers should share weights based on how much loss increases when one layer replaces another.
Critical strengths
The paper is very well written, clearly structured, and easy to follow. The motivation and methodology are explained clearly. The experiments are comprehensive, covering different model architectures, compression budgets, and several baselines.
Critical weaknesses
-
Practical benefits remain unclear. The compressed model still exhibits a performance gap compared with the original teacher. For example, on Qwen3.5-2B at B67, the test NLL remains at 3.238 after 5M recovery tokens, compared with 2.594 for the original model. Moreover, despite reducing the parameter count, decoding throughput decreases from 24.3 to 21.9 tokens/s. The method also introduces additional computational costs for functional substitution measurements, assignment search, and recovery training.
-
The downstream evaluation is restricted to benchmarks, primarily assessing language understanding and multiple-choice commonsense reasoning. lack of open-ended generation and challenging reasoning evaluations. Table 3 does not report the original teacher model's performance on these benchmarks. Without this reference, it is difficult to determine how much capability is actually retained after compression.
Questions for authors
-
what are the practical advantages of SubShare over directly using a smaller pretrained model?
-
Why are the downstream evaluations limited to likelihood-based benchmarks? Could the authors include open-ended generation tasks and report the original teacher's performance to better assess capability retention?
I maintain my original score.