- Rating
- 1
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 1
Read review
Paper summary
This paper introduces scPACK, a framework for compressing the scGPT foundation model using knowledge distillation. To this end, the authors propose sparse dictionary-based gene embeddings, backbone width and depth reduction, and Nash-MTL optimization. The evaluation shows that the compressed model performs competitively with scGPT on downstream tasks, including cell-type annotation, clustering, and donor mixing, while improving masked-expression reconstruction.
Critical strengths
- The research question is clearly stated.
- The manuscript is Easy-to-read (although the figures need more polish).
Critical weaknesses
-
The main problem is not clearly defined in the introduction. Existing distillation methods may align teacher and student representations without preserving local neighborhood relationships. Is this the limitation the authors aim to address? Why is preserving gene neighborhood biologically important, and how does the proposed method address this issue? These points should be clarified while providing supporting evidence and a discussion of why previous works could not address it.
-
No comparison with related approaches is reported.
-
scKAN, a potentially strong and reproducible competitor in this field, is not included, although it also distills knowledge from scGPT and reports better performance than it on downstream annotation tasks.
-
MicroSC, mentioned as a direct competitor, appears to have reproducibility issues due to the unclear definition of its distillation targets.
-
The ablation studies are insufficient. The effects of PCA initialization, the advantage of Nash-MTL over alternative optimization methods, and the individual contributions of the five proposed objectives are not investigated.
-
Table 2 shows that a simple PCA representation performs competitively with both teacher and student models on the cell-type annotation task. This raises questions about the added value of the learned representations compared with a simple baseline. I wonder why PCA is not also included in the evaluation of other tasks.
-
Donor mixing is mentioned in the abstract but is not sufficiently discussed. Does it refer to batch effects? If so, how do the authors identify the batch effect? Does the proposed method have the capacity to address it to some extent? If so, how?
-
Figure 6 seems relevant to this topic, but the representations are colored by cell type. Additional UMAP visualizations colored by gene expression and donor ID would be helpful.
-
Despite the substantial parameter reduction, the students consume more GPU memory than the teacher at larger batch sizes. This is an important limitation for practical deployment and should be explicitly discussed in the paper in a "limitations" subsection.
-
The figures need more polish. For example, in Figure 2, some axis labels are unclear, and the graphs overlap or are difficult to distinguish in some subfigures.
ref: "scKAN: interpretable single-cell analysis for cell-type-specific gene discovery and drug repurposing via Kolmogorov-Arnold networks." He, Haohuai, et al. Genome biology (2025)
Questions for authors
-
In the loss design, representations generated at internal layers of the model are compared. How were these layers selected?
-
The final GT-only stage substantially weakens prediction matching with the teacher, nearly eliminating the difference between fixed and "dynamic weighting". How does this affect the claimed benefit of Nash-MTL in the final deployed model?
-
Most main results and ablation studies are reported without variability estimates or statistical significance tests. Were experiments repeated with multiple seeds to assess the robustness and significance of the reported improvements?
Dear Reviewer,
Thank you for your detailed review and helpful suggestions on the experimental evaluation and presentation. Unfortunately, due to more time is needed to complete the remaining experiments and revise the paper, we have to decide to withdraw this submission. We appreciate the time and care you devoted to reviewing our work.
Best Regards