acceptodds
Under review as a conference paper at ICLR 2027

scGANT: Cell-Conditioned Biological Guidance for Single-Cell Foundation Models

Abstract

Single-cell foundation models learn transferable cellular representations from large-scale transcriptomic atlases (e.g. scRNA-seq). Most of these Transformer-based foundation models represent each cell as a sequence of gene tokens and rely on data-driven attention to infer gene dependencies from raw data. However, this principle is fundamentally mismatched to single-cell biology, where gene dependencies are biologically structured and cell-state specific. To model such cell-specific gene dependencies, we introduce scGANT, a novel single-cell foundation model framework with cell-conditioned biological guidance. scGANT includes two key innovations: (i) Cell-conditioned prior induction is designed to adapt multi-channel biological priors (including transcription-factor regulation, pathway co-membership, co-expression, and chromosomal proximity) to the current cellular context. (ii) The resulting cell-conditioned prior signals are then incorporated into the biologically guided attention, which determines which gene pairs are allowed to communicate and how strongly their interactions are weighted. Across five benchmark datasets spanning cell-level and gene-level tasks, scGANT improves macro-F1 on cell type annotation by up to 12.2%, achieves the SOTA frozen-encoder batch-integration overall scIB score, and the best performance on all four differential-expression metrics over 22 held-out perturbation predictions. In an additional case study, scGANT identifies a 12-gene Leiden cluster containing five genes with no direct prior edges among them. Published studies suggest that these genes are involved in related processes. This illustrates scGANT's hypothesis generation beyond direct prior-edge copying. Collectively, these results support biological guidance of gene-token communication as a promising design principle for single-cell foundation models. Code is available at: https://anonymous.4open.science/r/scGANT-1BFB.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
AI for Science, Single Cell

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_SvxeOpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
1
Soundness
3
Read review
Paper summary

This paper proposes scGANT, a single-cell foundation model framework that incorporates biological prior knowledge into the attention mechanism rather than treating it solely as input-side token features. The framework has two main components: (1) a cell-conditioned prior induction module that modulates the relative weights of four biological prior channels; and (2) a biologically-guided attention mechanism that combines this signal with representation-based similarity to sparsely route and weight gene–gene information exchange. Evaluation covers three downstream task families: cell-type annotation, frozen-encoder batch integration, and perturbation prediction.

Critical strengths
  1. The observation that gene dependencies are biologically structured and cell-state-specific is sound, and the paper's central thesis that priors should modulate information exchange between gene tokens rather than merely enrich token embeddings is a legitimate and under-explored design axis. The multi-channel prior construction is more comprehensive than most prior work.
  2. Unlike many single-cell foundation model papers that evaluate only on cell-type annotation or embedding tasks, this paper attempts to validate on perturbation prediction, which probes whether the model has learned functional gene-response structure.
Critical weaknesses
  1. Novelty is substantially undermined by closely related concurrent work, and the related-work treatment is inadequate. GRNFormer [1] already proposes a "biologically-guided framework" that integrates gene regulatory networks into RNA foundation models. The substantive distinction between scGANT's cell-conditioned prior induction and GRNFormer's adaptive cross-attention over cell-specific GRNs is not clearly articulated, let alone empirically compared. Given that GRNFormer is the most directly related prior work, its absence from the experimental comparison and the inaccurate characterization in Related Work are material concerns. Additionally, more recent foundation models such as CellFM [2], and TranscriptFormer [3] are not compared, despite being published or available before submission.
  2. scGANT reports perturbation results comparing only against GEARS, PRESAGE, Geneformer, scPRINT, and scGPT, none of which include a mean, linear, or "no-change" baseline. Without these controls, it is impossible to determine whether scGANT's reported gains reflect genuine biological learning or merely relative positioning among models that all underperform trivial baselines.
  3. Writing quality and notation are below the standard expected for a conference submission. The manuscript exhibits numerous issues consistent with insufficient human revision: notation is introduced without definition; intermediate variables proliferate without clear relational exposition.

References [1]GRNFormer: A Biologically-Guided Framework for Integrating Gene Regulatory Networks into RNA Foundation Models [2]CellFM: a large-scale foundation model pre-trained on transcriptomics of 100 million human cells [3]TranscriptFormer: A generative cell atlas across 1.5 billion years of evolution

Questions for authors

The revision suggestions is the same as that in Critical Weaknesses.

Reviewer_mX7YOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper introduces scGANT which aims to establish generalizable cell and gene embeddings by cell-condition prior induction and a biologically guided attention mechanism. Across 5 datasets, the model improve on cell and gene level tasks such as cell type classification, batch correction, perturbation prediction, and gene network analysis.

Critical strengths

I very much like the direction that the authors are taking where they try to make single cell foundation models change attention patterns based on prior graphical information between gene tokens.

Critical weaknesses
  1. i am not convinced by the ablation studies. The ablation study has no uncertainty estimates and the numbers across all cases are too similar, which fails to justify why all new components are required. Additionally, the setting of static priors vs. cell conditioned prior are missing, which actually leads to my next concern.

  2. I am not sure why each individual cell needs to change graphs. Regulatory relationships are often inferred at a cell type level rather than individual cell level. Additionally, if a gene–gene correlation network changes, that does not mean that regulatory relationship actually changes. The same applies to the attention-routing graph used for prediction. All experiments in the paper are cell type level anyway, so why do we need a cell conditioned prior?

Questions for authors
  1. I think a base case comparison is required to show that the model design is justified. For example, what if i compare with a model with with an integrated static graph across channels, matched neighbor budget, ordinary masked attention, and the same RWSE? If that achieves similar performance a lot of the extra operations are not needed.

  2. Some of the newer foundation models such as UCE(for example the protein embeddings and the chromosome information) and STATE tries to address the issue of biological information awareness in the model architecture. Why does the author think that graphical structure adds additional benefits? Can you compare to them?

Reviewer_nK1WOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper introduces scGANT, a single-cell foundation model that incorporates transcription-factor regulation, pathway membership, co-expression, and chromosomal proximity into gene-token processing. Cell-conditioned weights modulate these priors to guide a sparse attention branch alongside conventional dense attention. The model is pretrained on 37.5 million human cells and evaluated on cell-type annotation, batch integration, and perturbation prediction, with an additional gene-program case study.

Critical strengths
  1. Incorporating context-dependent biological knowledge into scFMs is an interesting and important direction.
  2. The results demonstrate substantial improvements and the evaluation covers both cell-level and gene-level tasks.
Critical weaknesses
  1. The perturbation evaluation lacks simple baselines, such as predicting the mean response across training perturbations (Figure 3).
  2. Generalization on unseen datasets is not fully demonstrated (Table2 and Table3). Although Table 2 evaluates disease-state and cross-study transfer, the authors do not clarify whether the evaluation datasets overlap with the pretraining corpus. In Table 3, the improvement on held-out covid 19 benchmark is marginal (0.681 vs. 0.677 for scGPT), with identical biological conservation scores.
  3. The benefit of cell-conditioned biological priors is not sufficiently established. There's a comparison between static and cell-conditioned priors only without BGA, but lacks a comparison between static and cell-conditioned priors within the full BGA architecture (Table 4 ).
Questions for authors
  1. See the weakness
  2. One important motivation for self-supervised scFM pretraining is learning biological structure directly from large-scale unlabeled data with minimal human bias, which might potentially allow discovering relationships absent from curated knowledge bases. From the perspective of the bitter lesson, could sufficiently large models trained on more diverse data learn these dependencies without manually introducing biological priors? How do the authors view the trade-off between incorporating established biological knowledge and preserving the ability to discover previously unknown biological relationships?

Comments on the paper

1 comment
AuthorsOpenReview ↗

Rebuttal Summary

Dear AC,

Thank you very much for the time and effort you have devoted to handling our submission. Below is a brief summary of our paper and the rebuttal for your consideration.

1. Strengths of the paper

  • A new way of using biological priors. Existing single-cell foundation models add biological knowledge to gene token features in a fixed way. Instead, scGANT is the only one whose biological priors are adaptively conditioned on a learned cell state and let priors guide how gene tokens exchange information in attention.
  • Two core designs. Cell-conditioned prior induction (CPI) adaptively reweights four prior channels by cell state, and biologically guided attention (BGA) uses them inside self-attention in every layer.
  • Broad evaluation. scGANT is pretrained on 37.5M cells and evaluated on five benchmarks across cell-level and gene-level tasks, together with a gene-program case study.

2. Strengths recognized by the reviewers

All three reviewers recognized the direction of our work. Reviewer nK1W described it as "interesting and important" and noted that "the results demonstrate substantial improvements and the evaluation covers both cell-level and gene-level tasks". Reviewer mX7Y wrote "I very much like the direction that the authors are taking". Reviewer Svxe called our thesis "a legitimate and under-explored design axis" and the multi-channel prior construction "more comprehensive than most prior work".

3. Main concerns addressed

  • Simple perturbation baselines: We added the no-change, mean, and linear baselines on Adamson. scGANT exceeds all three baselines on all four metrics.
  • Ablations: We reran the ablations with uncertainty estimates and reorganized them into two tables. Table A shows that both modules contribute: adding CPI and then BGA each improves the results. Table B shows that, within the full model, the cell-conditioned prior improves all four metrics over the static prior, including MS macro-F1 by 0.048 and COVID-19 Overall by 0.036. The full model also improves every metric over the base case requested in the review.
  • Why condition on each cell: We clarified that scGANT does not infer cell-specific GRNs but performs cell-state-conditioned prior fusion, and revised the wording in the paper. We also showed that cell-type labels explain only 37% of the variation in TF-program activity, and the remaining 63% lies among cells of the same type.
  • Comparison with more baselines: We compared with CellFM, TranscriptFormer, UCE, STATE-SE, and a reimplementation of GRNFormer on all three tasks. Against all of them, scGANT is best on all four MS annotation metrics and has the highest Overall on both integration datasets. These comparisons were added to the revised paper.
  • Novelty clarification: How scGANT differs from these models is now stated in the Introduction and Related Work and summarized in Appendix F (Table 11). Among these models, scGANT is the only one whose biological priors are adaptively conditioned on a learned cell state and act inside self-attention.

The revised manuscript has been uploaded, with all changes marked in purple.

4. Status of the discussion

Reviewer nK1W confirmed that the additional experiments and clarification that the response have resolved the concerns. After our first response, Reviewer Svxe had two remaining concerns, and we responded to both in detail and further revised the paper accordingly. We have also responded in detail to each of Reviewer mX7Y's initial comments.

Thank you again for your time and consideration!

Best regards,
The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.