acceptodds
Withdrawn from ICLR 2027

From gene programs to biological language: rethinking single-cell foundation models as modality translators

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Single-cell foundation models effectively capture gene expression patterns and cellular heterogeneity to support downstream analysis, but struggle to incorporate broader knowledge. Large language models possess such knowledge, yet cannot directly interpret single-cell expression data. Connecting these complementary capabilities often requires specialized model designs or modifications to existing frameworks. We introduce CellDialect, a framework that repurposes pretrained single-cell foundation models as modality translators. CellDialect converts expression profiles into cell-conditioned continuous tokens compatible with a frozen language model, without pretraining a new language backbone or changing its architecture. Joint numerical and language supervision supports cell representation learning, question answering, descriptive text generation, and expression generation conditioned on cellular metadata. Benchmark evaluations demonstrate improved representation quality over the evaluated baselines and competitive zero-shot cell task question answering. CellDialect connects existing single-cell and language foundation models within a shared framework. It offers a path to supporting diverse single-cell tasks and improving existing models with modest additional pretraining.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
Single-cell foundation models, Large language models, Modality translation, Multimodality

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_QPW2OpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

The paper introduces CellDialect, a framework that connects pretrained single-cell foundation models with frozen large language models (LLMs). It uses 16 learnable query tokens inserted into the scGPT encoder to obtain cell-conditioned representations, which are projected into the LLM embedding space. A dedicated representation token (REP) is used for numerical prediction, while language generation is trained through cell-type, tissue, and disease-related question answering and descriptive text generation. The framework also supports metadata-conditioned gene-expression generation.

The authors evaluate CellDialect on ten held-out datasets, reporting improvements in average NMI and ARI over several existing single-cell foundation models, as well as competitive performance on cell-related question answering. Although the framework is functional and the application is relevant, I have substantial concerns about its methodological novelty, the attribution of its reported improvements, and the adequacy of its experimental validation. In particular, the paper does not establish that its proposed cellular interface offers meaningful advantages over simpler approaches built directly on pretrained single-cell representations.

Critical strengths
  1. Connecting pretrained single-cell foundation models with general-purpose LLMs is an interesting research direction. CellDialect provides a unified framework for supporting both numerical and language-based tasks while keeping the LLM backbone frozen, which could be useful if its advantages over simpler alternatives were established.

  2. The authors evaluate the framework on ten held-out datasets spanning multiple tissues and include representation learning, question answering, and metadata-conditioned expression generation. This provides an initial assessment of the framework's applicability beyond a single dataset or task, although the current experiments do not sufficiently establish the unique contribution of the proposed architecture.

Critical weaknesses
  1. Limited methodological novelty. CellDialect combines established components, including query tokens, a pretrained scGPT encoder, an MLP projection, and a frozen LLM, without demonstrating a sufficiently distinctive methodological contribution. Key architectural choices are not justified: the authors do not compare query representations with cross-attention-based alternatives or simple pooling instead of the REP token. More importantly, they lack a direct baseline using scGPT representations with a simple prediction head, making it unclear whether the reported improvements come from the proposed LLM integration or primarily from fine-tuning the pretrained cell encoder.

  2. Insufficient experimental evidence. The ablations do not adequately isolate the contributions of individual components. The comparison between 8B and 1.3B LLMs is confounded by differences in pretraining and instruction tuning. Furthermore, QA tasks mainly cover conventional cell annotations, while expression generation is supported primarily by qualitative visualizations, without rigorous quantitative comparisons against simpler conditional-generation baselines. Overall, the experiments demonstrate that the framework can perform several tasks, but do not establish that the proposed architecture is necessary or offers substantial advantages over simpler alternatives.

Questions for authors
  1. Can the authors compare CellDialect with a simple scGPT-based baseline using a linear probe or MLP prediction head, as well as a model trained with the same reconstruction objectives but without the LLM? These comparisons are essential to determine whether the proposed integration provides benefits beyond fine-tuning the pretrained cell encoder.

  2. What justifies the use of 16 query tokens and a separate REP token? Can the authors compare this design with alternative aggregation strategies, such as mean pooling of query representations or cross-attention-based interfaces? The current inference-time token-removal ablation does not establish that the proposed design is necessary or optimal.

  3. Can the authors provide quantitative evaluations of metadata-conditioned expression generation against established conditional-generation baselines, particularly for recovering marker-gene expression and cell-type-specific expression programs? The current qualitative visualizations provide insufficient evidence of biological fidelity.

  4. How do the authors justify the choice of LLM backbone based on a comparison between Llama-3.1-8B-Instruct and Sheared-LLaMA-1.3B, given their differences in model size, pretraining, and instruction tuning? Can the contribution of backbone choice be evaluated in a more controlled setting?

Reviewer_P4NVOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes CellDialect, a multi-modal single-cell foundation model (scFM) that pairs an existing pre-trained scFM (scGPT) as a modality encoder for cells with a frozen large language model (LLM) backbone to enable multi-modal tasks. CellDialect is trained by freezing the LLM backbone, initialized from Llama-3.1 8B Instruct, and only training the initialized modality encoder and adaptation layer projecting scGPT output embeddings into the LLM’s input space. The authors evaluate CellDialect on embedding representation quality, predictive question answering, and conditional expression generation tasks, demonstrating that CellDialect has versatile task capabilities across language and single-cell tasks.

Critical strengths
  1. Model architectural choices: The proposed architecture makes use of strong architectural components with clear intensions: a pre-trained scGPT model serves as a strong modality-specific encoder, and a frozen Llama 3.1 model backbgone serves as a language decoder head. These constituent parts, along with decoding heads for both language and expression, give the model the potential to operate on language and expression-based inputs and outputs.
Critical weaknesses
  1. The proposed model architecture lacks significant task capabilities and evaluations beyond what current single-cell foundation models can achieve. All current single-cell foundation models are able to generate cell embeddings (benchmark 1 on representation quality) and do gene imputation or expression generation of some sort (benchmark 3 on metadata-conditioned expression generation). As examples, conditional scVI variants, scDiffusion, and C2S-Scale are a few models which are capable of doing metadata-conditioned expression generation in some format, and none of them are included in the benchmarking in Section 4.5. For the question answering benchmarking in Section 4.4 (Tables 2 and 3), the authors only evaluate on predictive questions related to cell type, disease, and tissue labels associated with cells. Nearly all single-cell foundation models are capable of predicting these labels in some capacity even without language capabilities, indicating that the QA benchmark proposed here does not evaluate language capabilities beyond simple prediction that current models can achieve.
  2. Novelty: The proposed model architecture lacks novelty in terms of going beyond its constituent parts from which it is initialized from. The modality-specific encoder, scGPT, is itself a foundation model which can complete nearly all of the tasks evaluated in this paper: cell embedding, label prediction for cell type, tissue, and disease, and gene expression imputation. The proposed addition of the frozen LLM backbone adds too little value over the modality-specific encoder relative to the additional compute overhead it brings in the current evaluated tasks.
  3. Missing baselines: The experimental evaluations for CellDialect are missing critical baseline foundation models, which are necessary to evaluate whether it makes meaningful improvements over the current field of single-cell foundation models. These include: (i) Geneformer and the STATE Embedding model for the representation quality benchmark (Table 1), (ii) closed-source LLMs such as GPT and Gemini using shortened cell language representations or differentially expressed genes as inputs for cells on the question answering task, and (iii) scDiffusion, conditional scVI variants, and C2S-Scale on metadata-conditioned expression generation.
Questions for authors
  1. The authors state in their Experimental Setup (Section 4.1) that the evaluated dataset IDs were excluded from CellDialect training, in order to evaluate zero-shot transfer to the 10 holdout datasets. Can the authors confirm if these 10 dataset IDs are also not present in the training data of the scGPT model used to initialize the encoder? Particularly for datasets released during and before 2023, when scGPT was released.
  2. In the mutual information representation quality benchmark (NMI), could the authors provide standard deviation or error bars to give an indication of variance in model predictions? That would help determine whether the performance gain of CellDialect relative to baselines and scGPT itself is significant.
Reviewer_rjqdOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

CellDialect connects scGPT to a frozen language model by converting gene expression profiles into continuous cell tokens. It jointly learns to reconstruct expression and generate text, supporting cell representations, attribute prediction, descriptions, and metadata-conditioned expression estimation. The approach is useful, but its added value beyond the underlying foundation models is not sufficiently established.

Critical strengths

The same interface supports representation learning, question answering, and numerical expression prediction.

Critical weaknesses
  1. Performance depends on the cellular information captured by scGPT and the knowledge and reasoning capabilities of the frozen LLM.

  2. The ablations mainly examine query-token removal and two language backbones. Comparisons against continued scGPT training with matched data and supervision, direct prediction heads, and simpler interfaces are needed to isolate the LLM’s contribution.

  3. Evidence is primarily qualitative UMAP placement and marker-expression comparisons. Quantitative comparisons against conditional-mean baselines and generative models are missing.

Questions for authors

How does performance change with different single-cell encoders, and which capabilities are contributed specifically by the frozen LLM?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.