acceptodds
Withdrawn from ICLR 2027

scPathVQ: A Pathway-Guided Discrete Latent Interface for Single-Cell Understanding and Generation

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Effective single-cell transcriptomic representations are fundamental to modeling cellular states in virtual cells. Existing methods typically tokenize individual genes, but are inherently constrained by the high-dimensional, sparse, and unordered nature of transcriptomic inputs. Moreover, cell embeddings are often obtained by flattening and pooling latent sequences, which can obscure the functional gene modules that collectively shape cellular states. We introduce **scPathVQ**, a pathway-guided discrete latent interface that provides a compact representation while preserving functional gene-module structure for both cellular-state understanding and generation. Its biology-structured tokenizer organizes transcriptome-wide genes into pathway-induced units that capture functional gene modules, complemented by highly variable genes that provide cell-specific expression context. A latent bottleneck compresses these contextualized features into a fixed-length sequence. Cell-state vector quantization maps each latent component to a shared codebook. The resulting quantized sequence preserves functional gene-module structure for perturbation response modeling, while gated pooling yields a compact cell level representation for downstream understanding tasks. We pretrain scPathVQ on 34 million human cells and evaluate several downstream tasks. The pooled representation supports strong transfer across cell-level tasks, while the quantized sequence enables accurate perturbation-response prediction. Generated expression profiles recover gene pathway responses and associated functional cell-state patterns. Ablation studies further identify the contribution of pathway-guided organization to the learned representation.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
Single-cell representation, Bioloigy-structured tokenization, Cell state VQ, Functional cell-state inference.

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_meUQOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes scPathVQ, a pathway-guided discrete representation learning framework for single-cell understanding and generation. The method organizes gene expression into pathway-induced patches complemented by highly variable gene tokens, integrates them through cross-attention, and compresses the resulting representations using learnable latent tokens and vector quantization. The quantized latent sequence supports perturbation response prediction, while a gated pooling mechanism produces continuous cell embeddings for downstream understanding tasks. The model is pretrained on approximately 34 million human cells and evaluated on cell-type annotation, batch integration, perturbation prediction, and functional cell-state inference. Experiments demonstrate competitive performance across these tasks, with additional ablations and pathway-level analyses investigating the proposed components.

Critical strengths
  • Unified representation for understanding and generation.
  • Strong performance across multiple benchmarks.
Critical weaknesses
  • Insufficient validation of pathway-guided tokenization. The proposed tokenizer flattens overlapping KEGG pathways, removes duplicate genes, and partitions the resulting sequence into fixed-size patches. Consequently, individual patches do not necessarily correspond to coherent biological pathways. Although removing pathway tokens reduces performance in Table 2, this ablation does not establish whether the gains arise from meaningful biological organization or simply from additional gene information and structured compression. A comparison with randomly grouped genes under a matched token budget would be important for supporting the central motivation.

  • Limited evidence for the necessity of discrete representations in generation. While the paper motivates vector quantization as a key component for preserving structured cellular states, the main component ablations in Table 2 focus on cell-type annotation. The perturbation experiments do not directly compare quantized sequences against their continuous latent counterparts or pooled representations under the same prediction architecture. Therefore, it remains unclear whether the reported generation improvements specifically require the proposed discrete latent interface.

Questions for authors
  • Could the authors clarify how the pathway-token ablation controls for differences in input gene coverage and representation capacity? Is there any comparison against random or otherwise biologically unstructured gene grouping with a matched number of patches?

  • Could the authors compare perturbation prediction using the quantized latent sequence against the corresponding continuous latent sequence, keeping the downstream prediction head unchanged? This would help establish the contribution of vector quantization to generation.

  • Could the authors clarify whether the downstream annotation datasets overlap with the pretraining corpus and whether the annotation splits are donor or study disjoint? This would help assess how strongly the reported results demonstrate generalization to unseen biological samples.

Reviewer_vDBGOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes scPathVQ, a single-cell foundation model that combines biologically structured gene tokenization, a multi-token latent bottleneck, and vector quantization (VQ) to support two kinds of tasks: cell understanding and perturbation-response prediction.

The architecture has four main elements:

  1. Pathway-guided tokenization: Organizes genes using KEGG pathways, forming fixed-size patches, together with a complementary set of highly variable genes (HVGs).
  2. Cross-view fusion and compression: Combines pathway-patch features with HVG features through cross-attention, then compresses the representation into 32 latent tokens.
  3. Vector quantization: Maps each latent token to a shared codebook of 512 discrete prototypes.
  4. Two downstream interfaces: Uses gated pooling to produce a cell embedding for annotation and integration, while retaining the quantized token sequence for perturbation-response prediction.

The model is pretrained on approximately 34 million human cells from CELLxGENE. The authors report strong performance on cell-type annotation, batch integration, and perturbation prediction, along with analyses of inferred pathway responses.

Critical strengths

The paper addressed the representation learning problem in single-cell foundation modeling. Several usage of techniques are interesting such as the usage of the vector quantization in the methodology and cell representation.

The experimental results are presented strong. scPathVQ reports strong performance on several cell-type annotation, batch integration, and perturbation prediction benchmarks. The evaluation covers multiple datasets and downstream tasks.

Critical weaknesses
  1. The novelty and contribution of the methodology is limited. The cell-state vector quantization, is a direct application of vq method into the application with minimal new modification. The authors discuss related methods such as CellVQ and scDiVa but do not provide direct experimental comparisons.

  2. Given many components introduced in the work, a more extensive ablation study should better tell the performance contributions. However in the current form, the table 2 and section 4.5 only ablate the components in the cell-type classification task setting. It is unclear whether these components contribute equally in other tasks, particularly the perturbation prediction task.

Questions for authors
  1. Have the authors compared the proposed pathway-guided patches against randomly grouped gene patches with the same gene coverage, patch sizes, and model capacity? This would help determine whether the gains come from the biological organization or from patch-based compression in general.

  2. Can the authors clarify whether any downstream evaluation datasets overlap with the CELLxGENE pretraining corpus, and how such overlap was handled? This is particularly relevant given the use of Cell Ontology labels during pretraining.

  3. For the pathway inspired representation, several other literature also exists. One example is TOSICA, published in 2023, arguably provides a more explicit pathway-level representation, where mapping from genes to pathway tokens is constructed by the gene membership in pathway. https://www.nature.com/articles/s41467-023-35923-4 . So how the work improves from existing efforts in representing pathway priors, is the proposed pathway-guided module theoretically and empirically better, or it can be replaced with existing ways of pathway prior implementations?

Reviewer_Gf6zOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes scPathVQ, a transformer model that tokenizes genes into pathway-guided patches, alongside highly variable gene tokens. The method then establishes a vector quantization bottleneck for cell representations that enables downstream tasks including representation (cell-type annotation, batch integration), generation (perturbation prediction), and interpretation (pathway inference). The downstream tasks are benchmarked against several well-known single-cell foundation models or more conventional baselines in the relevant field.

Critical strengths
  • The general idea of using pathways as units for representing a cell is sound, as well as using highly variable genes for cross attention.
  • The authors evaluate the model on a wide range of downstream benchmarks and compare with multiple well-known baselines.
Critical weaknesses
  • KEGG pathways differ substantially in size, and individual genes can belong to many pathways. Hence the patching can split one pathway across several patches or combine genes from different pathways within one patch. Although grouping genes by shared pathway memberships may help, how well the resulting patches preserve pathway structure needs further validation.

  • Several benchmarks need clarification. For the cell-type annotation benchmark, the paper does not clearly state how the baselines are adapted. For perturbation prediction, the handling of baselines and gene coverage for output gene panel and target gene panel also need clarification. For Figure 4, the authors claim that the PCA projections show "central batch overlap" and that the "results support improved batch mixing". However, several cell type regions in panel (b) appear segregated by batch in panel (a).

Questions for authors
  1. Pathway information: An ablation for pathway information using random gene grouping with matched patch sizes and model configurations would be useful. The paper describes retaining the first appearance of each gene when laying out the pathways' genes in a fixed order. How sensitive are the results to shuffling pathway order and gene order within pathways?

  2. Cell type annotation: In Table 1, are the foundation model baselines fine-tuned on the training split by jointly optimizing their encoders and classifiers, as for scPathVQ^(f)? Please clarify the classifier and training protocol used for each method.

  3. Perturbation gene panels: Given that pretraining uses the union of pathway genes and HVGs (n=9159), how are the response gene panel and the perturbation target panel defined, and how are genes outside this union handled? What is their coverage relative to the original Adamson, Norman, and Dixit studies? Please also clarify how gene coverage is handled for the baselines.

  4. Batch integration: UCE is a well-known method for zero-shot cell embeddings, and is included in the cell-type annotation benchmark. Including it here would strengthen the comparisons. Also, could the authors provide evidence of batch mixing within cell types to support the interpretation of Figure 4? Furthermore, the paper describes the evaluation as scIB-style, but Equation (29) uses only 1−ASWbatch1-\mathrm{ASW}_\mathrm{batch}, leaving other scIB batch metrics absent. Could the authors report additional scIB batch metrics, such as kBET and iLISI, to support the batch integration conclusions?

  5. Figure 6: What cells are visualized in the UMAP, and how are their coordinates assigned across the rows? Points appear to repeat between ("Pre-perturbation", "Cell cycle") and ("Ground truth", "Apoptosis"). If only control cells are shown, how are the "Ground truth" pathway states assigned back to them, given that control and perturbed cells are unpaired?

  6. Figure 12: Searching "Harmony" reveals invisible text saying, e.g., "Melanoma cohort - Harmony - Batch", although the results are in support of the integration performance of scPathVQ. Please clarify the methods used to produce these figures.

  7. Figure 14: Several predicted pathway responses appear more uniform across perturbations than the observed responses. Could the authors quantify how well the model captures differences between perturbations, beyond the correlations pooled across all pathway-condition pairs?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.