- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 1
Read review
Paper summary
The paper introduces CellDialect, a framework that connects pretrained single-cell foundation models with frozen large language models (LLMs). It uses 16 learnable query tokens inserted into the scGPT encoder to obtain cell-conditioned representations, which are projected into the LLM embedding space. A dedicated representation token (REP) is used for numerical prediction, while language generation is trained through cell-type, tissue, and disease-related question answering and descriptive text generation. The framework also supports metadata-conditioned gene-expression generation.
The authors evaluate CellDialect on ten held-out datasets, reporting improvements in average NMI and ARI over several existing single-cell foundation models, as well as competitive performance on cell-related question answering. Although the framework is functional and the application is relevant, I have substantial concerns about its methodological novelty, the attribution of its reported improvements, and the adequacy of its experimental validation. In particular, the paper does not establish that its proposed cellular interface offers meaningful advantages over simpler approaches built directly on pretrained single-cell representations.
Critical strengths
-
Connecting pretrained single-cell foundation models with general-purpose LLMs is an interesting research direction. CellDialect provides a unified framework for supporting both numerical and language-based tasks while keeping the LLM backbone frozen, which could be useful if its advantages over simpler alternatives were established.
-
The authors evaluate the framework on ten held-out datasets spanning multiple tissues and include representation learning, question answering, and metadata-conditioned expression generation. This provides an initial assessment of the framework's applicability beyond a single dataset or task, although the current experiments do not sufficiently establish the unique contribution of the proposed architecture.
Critical weaknesses
-
Limited methodological novelty. CellDialect combines established components, including query tokens, a pretrained scGPT encoder, an MLP projection, and a frozen LLM, without demonstrating a sufficiently distinctive methodological contribution. Key architectural choices are not justified: the authors do not compare query representations with cross-attention-based alternatives or simple pooling instead of the REP token. More importantly, they lack a direct baseline using scGPT representations with a simple prediction head, making it unclear whether the reported improvements come from the proposed LLM integration or primarily from fine-tuning the pretrained cell encoder.
-
Insufficient experimental evidence. The ablations do not adequately isolate the contributions of individual components. The comparison between 8B and 1.3B LLMs is confounded by differences in pretraining and instruction tuning. Furthermore, QA tasks mainly cover conventional cell annotations, while expression generation is supported primarily by qualitative visualizations, without rigorous quantitative comparisons against simpler conditional-generation baselines. Overall, the experiments demonstrate that the framework can perform several tasks, but do not establish that the proposed architecture is necessary or offers substantial advantages over simpler alternatives.
Questions for authors
-
Can the authors compare CellDialect with a simple scGPT-based baseline using a linear probe or MLP prediction head, as well as a model trained with the same reconstruction objectives but without the LLM? These comparisons are essential to determine whether the proposed integration provides benefits beyond fine-tuning the pretrained cell encoder.
-
What justifies the use of 16 query tokens and a separate REP token? Can the authors compare this design with alternative aggregation strategies, such as mean pooling of query representations or cross-attention-based interfaces? The current inference-time token-removal ablation does not establish that the proposed design is necessary or optimal.
-
Can the authors provide quantitative evaluations of metadata-conditioned expression generation against established conditional-generation baselines, particularly for recovering marker-gene expression and cell-type-specific expression programs? The current qualitative visualizations provide insufficient evidence of biological fidelity.
-
How do the authors justify the choice of LLM backbone based on a comparison between Llama-3.1-8B-Instruct and Sheared-LLaMA-1.3B, given their differences in model size, pretraining, and instruction tuning? Can the contribution of backbone choice be evaluated in a more controlled setting?