acceptodds
Under review as a conference paper at ICLR 2027

TransMap: Metric Embedding of Gene Relational Structures for Image-Native Single-Cell Learning

10 views

Abstract

In single-cell omics, gene expression profiles lack canonical spatial coordinates, yet exhibit relational structure that can be conserved across species. We propose an image-native representation scheme that incorporates this structure into the input geometry for single-cell learning. The TransMap layout maps gene relationships onto a two-dimensional grid using Gromov–Wasserstein (GW) transport, with fused GW incorporating soft ortholog correspondences across species. Each selected gene occupies its own pixel, so its expression value is preserved exactly, while gene–gene distances are preserved approximately. We develop TransMap, a masked self-supervised model with a shared encoder, to learn from these images across species. Across five single-species datasets, the layouts achieve competitive distance and neighborhood preservation, while analysis across five species shows corresponding spatial organization of conserved modules. In a layout ablation across five datasets and two model sizes, GW layouts achieve lower mean masked patch reconstruction error than random layouts in all ten comparisons. Using unified human–mouse layouts constructed from training data alone, we pretrain a shared model on approximately 732K training metacells. On two of three human benchmarks, zero-shot scores fall within the range of evaluated foundation models; the same encoder also yields cell embeddings on two mouse datasets. These results support image-native representation as a promising basis for cross-species foundation models and motivate further study at larger data and model scales.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
Single-cell transcriptomes, Optimal transport, Gromov-Wasserstein distance, Metric embedding, Masked patch prediction, Self-supervised pretraining, Cross-species alignment

Reviews

Among:
Shape:
1234this paper, Top 32%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.50 over 4 reviews, above 68% of the others.
Reviewer_CLUfOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces TransMap, which converts single-cell gene expression profiles into two-dimensional images organized by biological relationships. Gene correlations and shared-nearest-neighbor geodesic distances define the relational structure, which is mapped onto distinct pixels through Gromov–Wasserstein transport and injective matching. For cross-species layouts, ortholog correspondences encourage related genes from different species to occupy nearby positions on a shared grid. The resulting images are used for masked self-supervised learning with a shared convolutional encoder, a local reconstruction head, and an expression prediction head operating on the cell embedding.

Critical strengths

Gene expression profiles have no natural two-dimensional coordinates. Using gene relationships to construct the input geometry provides an interpretable spatial prior for convolutional models. Assigning each selected gene to a separate pixel also avoids the loss of expression values caused by merging multiple genes.

The two-channel representation on page 4 distinguishes expression values from gene availability, and the loss is computed only over masked, available genes. This design accounts for differences in gene coverage across data sources.

Critical weaknesses

A central component of the method is the use of orthology to align input geometry across species. However, Table 2 on page 8 evaluates human and mouse datasets separately and does not test cross-species cell correspondence in the shared embedding space. The ability to process both species with one encoder does not establish that the model has learned transferable biological representations.

Equations (13) and (17) on pages 20–21 impose exact marginal constraints for both genes and pixels. In contrast, page 23 states that the human–mouse implementation enforces only the gene marginal exactly and uses a KL penalty with weight 1 for the pixel marginal. These are different optimization formulations. The authors should provide the objective actually used and clarify how it relates to the stated equations.

Questions for authors

Does the joint FGW layout improve cross-species cell retrieval or label transfer compared with independently constructed layouts? Could the authors provide controls without orthology constraints and with single-species pretraining?

Does TransMap still outperform random layouts, DeepInsight, and IGTD when the gene-level masks, model architecture, and parameter budget are held fixed?

5 comments
AuthorsOpenReview ↗

W1 + Q1: cross-species transfer

W1. A central component of the method is the use of orthology to align input geometry across species. However, Table 2 on page 8 evaluates human and mouse datasets separately and does not test cross-species cell correspondence in the shared embedding space. The ability to process both species with one encoder does not establish that the model has learned transferable biological representations.

Q1. Does the joint FGW layout improve cross-species cell retrieval or label transfer compared with independently constructed layouts? Could the authors provide controls without orthology constraints and with single-species pretraining?

Response:

Thank you for this question. We ran the single-species control, and it shows that the representations transfer across species: a model pretrained on human data alone embeds mouse cells better than any other zero-shot model we evaluated. This model has the architecture and objective of the joint model and was trained on the human metacells only, matching the number of human-data updates in the joint run (28,360 steps, against 28,600). It has never seen a mouse cell.

Zero-shot modelHypoMap (mouse)Pancreas (mouse)
TransMap, pretrained on human only0.4450.599
TransMap, pretrained on human and mouse0.4190.571
UCE0.4360.550

(Overall scores, as in Table 2.)

The human-only model has the highest score on both mouse datasets, above the jointly pretrained model and above UCE, whose pretraining data include mouse. Its biological conservation on the mouse data equals that of the joint model (0.671 and 0.857, against 0.687 and 0.841), and its lead in the overall score comes from batch correction (0.105 and 0.212, against 0.016 and 0.167): a model that has never been trained on mouse data has not fit the technical variation of mouse datasets, so mouse batches mix better in its embedding.

An encoder that has only seen human expression images reads mouse cells because the unified layout puts each mouse gene at or next to the pixel of its human ortholog: half of the 17,108 high-confidence ortholog pairs share a pixel, and 64% lie within one pixel. The orthology-aligned geometry carries the transfer. Joint pretraining adds biological conservation: averaged over the five datasets it reaches 0.700, against 0.661 for the human-only and 0.572 for the mouse-only model, and the overall score is 0.490 against 0.472 and 0.431.

Appendix G describes the single-species models, and Figure 9 gives their scores on every dataset.

AuthorsOpenReview ↗

W2: marginal constraints

W2. Equations (13) and (17) on pages 20–21 impose exact marginal constraints for both genes and pixels. In contrast, page 23 states that the human–mouse implementation enforces only the gene marginal exactly and uses a KL penalty with weight 1 for the pixel marginal. These are different optimization formulations. The authors should provide the objective actually used and clarify how it relates to the stated equations.

Response:

Thank you for the careful reading; the reviewer is right that the two descriptions did not match. In Equations (13) and (17) we reproduced the standard textbook formulation of entropic (fused) Gromov–Wasserstein, with the coupling constrained to Π(μ,ν)\Pi(\mu,\nu), without adapting it to our setting.

In our setting the grid has more pixels than genes, so exact marginals are neither appropriate nor what we solve: the implementation penalizes deviations from the two marginals with Kullback–Leibler terms of weight ρ=1\rho=1, which allows pixels to remain unoccupied and gives the solver more freedom to arrange genes.

This was an error in the write-up and does not affect any experiment: the layouts were computed with this relaxed objective throughout, so no result changes. We have replaced the constraint in Equations (13) and (17) with the KL-penalized objective, so that the stated equations now follow the implementation, and aligned the wording of Appendix E (page 23) accordingly. The one-gene-per-pixel property of the final layouts is unaffected, since it is enforced by the sparse matching step applied to the soft coupling.

AuthorsOpenReview ↗

Q2: layouts under fixed masks, architecture and parameter budget

Q2. Does TransMap still outperform random layouts, DeepInsight, and IGTD when the gene-level masks, model architecture, and parameter budget are held fixed?

Response:

The comparison with random layouts holds the architecture, the parameter budget, the data and the patch-masking scheme fixed, and the GW layout wins at every model size. In Table 1 the two layouts are trained with the same model at each size (458K and 15M parameters), on the same genes and data, and the GW layout has the lower reconstruction error in all ten comparisons, by 2.4–14.3%. At the scale of the final model, we pretrained three 147M-parameter models on random layouts with the corpus, architecture, optimizer and schedule of the main run. The GW layout raises the biological-conservation score of the zero-shot embeddings from 0.620 ± 0.022 to 0.700, with a gain on each of the five datasets, and the overall score from 0.441 ± 0.014 to 0.490 (Appendix G, Figure 8).

Masks are drawn on patches, so the set of masked genes follows the layout, and this favors the random layout. In the GW layout co-expressed genes sit in the same patch, so masking a patch removes a whole module at once: 20–33% of a gene's ten most correlated genes share its patch, against about 3% in a random layout, and the most similar gene that remains visible is less correlated with the masked one on every dataset (Appendix D.2, Figure 7). The GW layout reaches the lower error on the harder task. The 147M-parameter models are compared without any masks: embeddings are computed from the unmasked expression profile, so both models see the same input.

For DeepInsight and IGTD, Table A1 compares the layouts directly, and ours preserves the ordering of gene–gene distances best on all five datasets (Spearman 0.72–0.74, against 0.53–0.62 and 0.38–0.48). The TransMap layout also does something these methods cannot. They arrange the features of a single dataset, whereas ours places the genes of several species on one grid, with orthologs at corresponding pixels. This is what enables shared pretraining and the human-to-mouse transfer in our response to W1.

Reviewer_mf9KOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes TransMap, which maps gene relationships onto a two-dimensional grid using Gromov–Wasserstein (GW) transport and fused GW with soft ortholog correspondences, assigning each selected gene to a distinct pixel. A shared convolutional encoder is pretrained on approximately 732K human and mouse metacells using local masked reconstruction and global expression prediction. GW layouts achieve lower reconstruction error than random layouts across five datasets and two model sizes, while zero-shot scores fall within the range of evaluated foundation models on two of three human benchmarks.

Critical strengths
  1. Encoding gene relationships into the spatial organization of otherwise unordered expression profiles is an interesting direction for single-cell representation learning.
  2. GW layouts consistently reduce mean reconstruction error relative to random layouts across five datasets and two model sizes, with standard deviations reported over repeated runs. This provides initial evidence for their utility in masked reconstruction.
  3. The authors acknowledge important limitations, such as, lower zero-shot scIB aggregate scores than all three evaluated foundation models on DKD.
Critical weaknesses
  1. The layout ablation only demonstrates improved reconstruction in smaller models; it does not establish better cell representations or downstream performance in the final 147M-parameter model. Comparisons among GW, random, and alternative layouts under matched training conditions are needed to isolate the contribution of the proposed geometry.
  2. The additional benefit of cross-species pretraining remains unverified. Processing human and mouse profiles with one encoder does not establish effective knowledge sharing without controlled single-species versus joint-pretraining comparisons or cross-species transfer evaluations.
  3. TransMap uses all available layout genes, whereas the official foundation-model entries use 2,000 HVGs, and HypoMap aggregate scores use different metric sets. These differences limit conclusions about relative model performance.
  4. Patch masking changes which genes are masked together when the layout changes. Reconstruction improvements may therefore reflect both spatial organization and differences in prediction difficulty. A comparison using identical masked gene sets would help isolate the layout’s contribution.
Questions for authors
  1. Under matched training data, masked gene sets, and comparable parameter counts, how would a gene-token Transformer or an MLP masked autoencoder perform on the Table 2 benchmarks?

  2. With other conditions fixed, how do downstream scIB scores change when TransMap layouts are replaced with random layouts? Could you compare independent randomization across species with a shared random pixel permutation that preserves existing same pixel correspondences? Smaller-scale controlled experiments would also be informative.

  3. If identical gene sets are masked before mapping them to each layout, does the TransMap layout still outperform random placement in reconstruction error and downstream embedding quality?

  4. With matched per-species sample updates, does joint pretraining outperform single-species pretraining? What happens when ortholog constraints are removed while keeping structural and regularization settings unchanged? Cross-species label transfer or cell-type alignment, potentially compared with SATURN/UCE, would provide further evidence.

  5. How do results change when the CLS loss or species vector is removed? What happens when the position-specific linear CLS readout, which accounts for approximately 58% of the parameters, is replaced with global or attention pooling?

  6. Could you provide the complete pretraining dataset list, including CELLxGENE dataset identifiers and versions, and explain how duplicate datasets, cell overlap, and donor overlap with the five benchmarks were assessed? If donor independence cannot be established, please clarify this limitation.

8 comments
AuthorsOpenReview ↗

Q6: dataset construction

Q6. Could you provide the complete pretraining dataset list, including CELLxGENE dataset identifiers and versions, and explain how duplicate datasets, cell overlap, and donor overlap with the five benchmarks were assessed? If donor independence cannot be established, please clarify this limitation.

Response:

We thank the reviewer for these questions. We have revised Appendices F, G and H to answer them and summarize the changes here.

The pretraining collections are now listed with their sources in Table A2. For each collection, the table gives the reference, the accession or release from which we obtained the data, the split (training, validation or test), and the numbers of units, cells and metacells. The cartilage collection pools 19 studies, and Appendix F gives the accession of each. We collected the pretraining data from the original releases of the atlases and studies, namely GEO, SRA, BioProject, figshare and the atlas portals. The identifiers and versions in Table A2 are therefore those of the original releases, for example the Tabula Sapiens v2 release and versions 1.1, 2.0 and 3.0 of the Mouse Cell Atlas. CELLxGENE identifiers apply to the five benchmark datasets, which follow OpenProblems and come from CELLxGENE; Appendix H.1 gives the CELLxGENE dataset identifier of each.

On duplicate datasets, the collections do not share cells (new paragraph "Overlap between collections" in Appendix F). The three Mouse Cell Atlas versions profile different cells, as do the droplet and plate-based parts of Tabula Sapiens and of Tabula Muris, and the studies curated by 3CA are not among the other collections. There is one case of redistribution: Tabula Muris Senis includes the 3-month cells of Tabula Muris, and we removed these cells from Tabula Muris Senis by age and cell identifier. Related releases are assigned to the same split, which keeps them on one side of the boundary between training and validation.

We assessed cell overlap with the five benchmarks by source study. The studies behind the five datasets (Steuernagel et al., 2022; Hrovatin et al., 2023; Domínguez Conde et al., 2022; Eraslan et al., 2022; Wilson et al., 2022), including the studies combined in the two integrated atlases, HypoMap and the mouse pancreatic islet atlas, are not among the pretraining collections. The benchmark cells therefore entered neither the gene layouts nor any split (Appendix H).

Donor independence across studies cannot be established from the public data, because each study assigns its own donor identifiers; Appendix G now states this as a limitation. The benchmarks come from studies outside the corpus, and pretraining uses no cell-type labels, so the benchmark labels remain unseen by the model whatever the donor overlap.

AuthorsOpenReview ↗

W4 + Q3: masked gene sets design and comparison

W4. Patch masking changes which genes are masked together when the layout changes. Reconstruction improvements may therefore reflect both spatial organization and differences in prediction difficulty. A comparison using identical masked gene sets would help isolate the layout's contribution.

Q3. If identical gene sets are masked before mapping them to each layout, does the TransMap layout still outperform random placement in reconstruction error and downstream embedding quality?

Response:

Thank you for this question. The advantage of the GW layout does not come from an easier masking task. We measured the prediction difficulty under both layouts, and the result strengthens the comparison: patch masking is harder under the GW layout, and the GW layout still has the lower error. In downstream embedding quality, where no gene is masked, the GW layout is ahead as well.

Masking related genes together is the purpose of the design. On the TransMap layout a patch is a group of co-expressed genes, so patch masking hides whole gene modules, at a scale set by the patch size, and the model has to predict a module from the other modules instead of copying a correlated gene that stays visible. A model over a gene vector or gene tokens has no such notion of locality and needs an external definition of gene sets to mask in this way; here the layout provides it.

Under a random layout, the genes correlated with a masked gene are scattered over the image and most of them stay visible. We quantified this in the setting of Table 1 (2,000 genes on a 48×48 grid, 22 of 36 patches masked per cell, the same layout files). For each gene we took its ten most correlated genes and asked how many of them are masked together with it, and how strongly the masked gene correlates with the most similar gene that remains visible.

DatasetCorrelated genes masked together with the gene (GW / Random)Best remaining clue for a masked gene, |r| (GW / Random)
Lung0.710 / 0.6110.358 / 0.396
Breast0.709 / 0.6110.287 / 0.320
CITE-seq PBMC0.679 / 0.6110.311 / 0.337
Gut Fetal0.704 / 0.6110.401 / 0.439
Lung Fetal0.731 / 0.6110.460 / 0.516

On every dataset a masked gene loses more of its correlated genes under the GW layout and is left with a weaker clue. The layout itself shows why: 20–33% of a gene's ten most correlated genes share its patch, against about 3% in a random layout. On this harder task the GW layout lowers the reconstruction error by 2.4–14.3% in all ten comparisons of Table 1.

For downstream embedding quality the masks play no role, because embeddings are computed without masking and both models receive the same input for every cell. There the GW layout raises the biological-conservation score from 0.620 ± 0.022 to 0.700, with a gain on each of the five datasets, and the overall score from 0.441 ± 0.014 to 0.490 (Figure 8b). The two heads of the pretrained model show where the gain arises. The CLS head, which predicts the masked genes from a single cell vector, reaches the same loss under both layouts (0.076 and 0.077), while the local decoder, which uses spatial neighborhoods, improves from 0.141 to 0.130 (Figure 8a).

Both parts of the question therefore have the same answer. The lower reconstruction error of the GW layout is not a product of its masks, which make its task harder, and its better embeddings cannot be one, because no gene is masked when they are computed. In both cases the gain comes from the spatial organization of the genes, which the convolutional encoder exploits.

We have added this analysis to Section 4.3 and Appendix D.2 (Figure 7), and Section 3.2 now states the purpose of patch masking.

AuthorsOpenReview ↗

Q5: position-specific readout, loss components and species vector

Q5. How do results change when the CLS loss or species vector is removed? What happens when the position-specific linear CLS readout, which accounts for approximately 58% of the parameters, is replaced with global or attention pooling?

Response:

Thank you for this excellent question! The parameter count is right: the readout holds 84.9M of the 147.1M parameters. Where the model spends its parameters matters a great deal to us, and we were glad to take this question as the occasion to examine this part of the architecture closely. We ran a series of experiments on it, all of them full pretraining runs: an attention-pooling readout, lighter position-specific readouts, and each loss alone. We have also added the motivation for the readout to Section 3.2, which the submission did not clearly state. In short, the flatten readout is the heaviest part of the model, it is there for a reason, and it gives the best scores; pooling saves these weights at a large cost in biological conservation.

The motivation lies in what a pixel means in an expression image. Every pixel is a fixed gene, so the same feature at two positions describes two different sets of genes, and a shifted image is a different cell. The cell representation is therefore not translation invariant, and the readout keeps a separate weight for every position of the deepest feature map.

This design has a history in computer vision. The early convolutional networks for natural images read out their features in the same way, by flattening the last feature map into fully connected layers, and that layer was their heaviest part: it holds 62% of the parameters of AlexNet (Krizhevsky et al., 2012) and 74% of VGG16 (Simonyan & Zisserman, 2015). Global average pooling (Lin et al., 2014) later replaced it and was adopted by GoogLeNet and ResNet. Pooling could remove the layer because the class of a natural image does not depend on where the object lies, so position can be pooled away. Expression images do not have this invariance, which makes the position-specific readout the appropriate one here, and the ablation confirms it:

ReadoutReadout parametersModel parametersBioBatchOverall
Flatten (submitted model)84.9M147.1M0.7000.1740.490
Attention pooling1.5M63.6M0.4910.2170.382
1×1 conv to 128 ch, then flatten21.3M83.5M0.6890.1800.486
1×1 conv to 32 ch, then flatten5.3M67.5M0.6590.1890.471

(Means over the five datasets, scored as in Table 2. Every row is a full pretraining run.)

Attention pooling removes almost all readout parameters, and it loses 0.21 in biological conservation and 0.11 in the overall score. The flatten readout has the highest biological-conservation and overall scores in the table, so its weights are put to use.

The last two rows show how to make the readout lighter while keeping it position-specific. A 1×1 convolution shared by all positions reduces the deepest feature map from 512 to 128 channels before it is flattened. Every position keeps its own weights, the readout shrinks by 75% and the model by 43%, and the overall score is 0.486 against 0.490. With 32 channels the readout has 5.3M parameters and the model scores 0.471, still far above attention pooling. The 1×1 convolution comes from the same line of work as global pooling (Lin et al., 2014), and it applies here because it leaves the positions intact.

Each loss contributes. The submitted model, trained with both, reaches 0.700 in biological conservation and 0.490 overall. With the CLS loss removed, the readout receives no gradient, so we take the first 512 principal components of the flattened deepest feature map as the cell representation, and the scores fall to 0.596 and 0.447. In the other direction, the CLS loss alone gives 0.653 and 0.466.

The species vector is a special case. In the present design it is equivalent to a bias. It enters only the linear prediction head, where it adds to the prediction of each gene an offset that depends on the species, and human and mouse genes have separate parameters (qg,bg)(q_g, b_g) in this head, so for every gene this offset coincides with its own bias bgb_g. The model without the species vector represents the same functions, the encoder and the readout do not receive it, and the embedding is computed without it. We keep it as an interface for later designs. When pretraining extends to more species, the species vectors can carry their evolutionary relatedness, with closely related species receiving similar vectors, and once gene parameters are shared between species, for example between orthologs, the species vector is what lets the shared parameters give species-specific predictions.

Section 3.2 now gives the motivation for the readout, and Appendix G its history and the readout and loss ablations. Appendix H.5 and Figure 9 give the scores of these models, including the overall score on each dataset.

Reviewer_cDs7OpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes a single-cell representation learning framework, TransMap. The authors construct distances based on co-expression relationships between genes and, through optimal transfer using a gate-grid, approximate these relationships into a two-dimensional grid. Each selected gene occupies an independent pixel, and the pixel value records its expression level. For cross-species modeling, the authors use a free gate-grid to simultaneously consider gene relationships within a species and orthologous relationships between species, ensuring that each species retains its independent gene set and fixed layout while encouraging homologous genes to be located close together. Subsequently, joint human-mouse pre-training is performed through a shared convolutional encoder and two masking tasks: local reconstruction and cell vector prediction. Experiments show that the GW layout achieves a lower average reconstruction error than the random layout in all ten comparisons. After joint human-mouse pre-training, the model reaches the score range of the base model being compared on two of the three human benchmarks and is able to generate cell representations for two mouse datasets. These results support the feasibility of single-cell representation learning based on image input, but do not yet demonstrate leading performance, nor have they verified the effectiveness of shared pre-training outside of humans and mice.

Critical strengths

1.Input representation with clear biological motivation. TransMap encodes gene co-expression relationships into a 2D layout, providing biologically meaningful neighborhood structures for convolutional models. Each selected gene occupies an independent pixel, thus preserving the input expression value; the formal pre-trained layout is constructed using only training data.

2.Flexible cross-species representation design. The method combines intra-species gene relationships with cross-species homology correspondences, supporting one-to-many and many-to-many homology relationships while preserving species-specific genes. This provides a unified input format for shared encoders to handle species with different gene sets.

3.Clear introduction of introduction and methods part. The paper clearly introduces the gene layout construction, shared encoder and dual-branch training objectives.

Critical weaknesses

1.The advantages of 2D embedding over directly utilizing gene relationships have not been fully established. Embedding gene relationships into a 2D grid can adapt to convolutional architectures, but it also introduces distortions in distance and neighborhood structure. While the expression values of selected genes are preserved, the relationship structure can only be approximated. Table A1 has evaluated layout fidelity, but the relationship between these metrics and representation quality is not fully explained, and there is a lack of model comparisons that directly utilize the same gene relationship graph. 2.The contribution of spatial layout to the final cell representation lacks sufficient validation. The model flattens the deepest feature maps and linearly projects them into a 512-dimensional cell representation. Does this flattening-and-projection step discard some of the spatial information encoded in the image? From the perspective, what advantage does this approach offer over directly processing the same gene-expression values in a fixed random order? 3.It is hard to judge performance of design: a)TransMap has not yet demonstrated a significant advantage over existing methods. It performs close to the comparison models on GTEx and falls below the three base models compared on DKD; self-supervised fine-tuning has also failed to consistently improve the overall score. b)Comparisons with existing representation visualization methods should be completed. Figure 3A only shows phenonmenons that not so related to the main story, instead of relationship with designs/performance。 Figure 4 shows only TransMap embeddings, providing limited evidence of its advantages over competing methods. Also, this figure is hard to read. c)TransMap uses all available layout genes, while other base model entries use 2,000 hypervariable genes; HypoMap's overall score also uses a different set of metrics. These differences limit the explanatory power of direct comparisons. It is recommended to standardize input and evaluation metrics and report the uncertainty of repeated runs, as well as the results for biological preservation and batch correction. d) The caption of Table 2 is overly verbose and redundant. The caption should be condensed to focus on details essential for interpreting the results and understanding the comparisons.

Questions for authors

1.Why choose a two-dimensional grid to represent gene relationships? What advantages are expected over learning directly on a gene relationship graph? Could the authors analyze how the distance and neighborhood preservation metrics in Table A1 relate to downstream cell representation quality? 2.Can the advantages of the TransMap layout be validated within the same learning framework? Could the authors compare the downstream performance of TransMap, a fixed random layout, DeepInsight, IGTD, and GenoMap while keeping the gene set, training data, encoder, training objective, and training budget fixed? 3.How do individual model components contribute to the quality of the final cell representation? Could the authors provide ablations of the readout mechanism and loss components to clarify the respective contributions of different parts of designs? 4.Do the main benchmark conclusions hold under matched evaluation conditions? Could the authors repeat the comparisons using matched input gene sets and identical metric sets, reporting biological conservation and batch correction scores separately, together with uncertainty across repeated runs? Could Figure 4 include baseline visualizations using the same UMAP settings, with additional plots colored by batch?

9 comments
AuthorsOpenReview ↗

Q2: Random layout

Q2. Can the advantages of the TransMap layout be validated within the same learning framework? Could the authors compare the downstream performance of TransMap, a fixed random layout, DeepInsight, IGTD, and GenoMap while keeping the gene set, training data, encoder, training objective, and training budget fixed?

Response:

Yes. A fixed random layout is the direct test within the same learning framework: it removes the gene relationships from the image and keeps everything else. We pretrained three 147M-parameter models on such layouts, with the gene set, corpus, encoder, objective, optimizer and number of steps of the main run, and scored their zero-shot embeddings with the same protocol. The GW layout raises the biological-conservation score from 0.620 ± 0.022 to 0.700, with a gain on each of the five datasets, and the overall score from 0.441 ± 0.014 to 0.490 (Appendix G, Figure 8). Embeddings are computed from the unmasked expression profile, so the two kinds of model receive identical inputs and differ only in where the genes sit.

For DeepInsight and IGTD, Table A1 compares the layouts directly, and ours preserves the ordering of gene–gene distances best on all five datasets (Spearman 0.72–0.74, against 0.53–0.62 and 0.38–0.48). The TransMap layout also does something these methods cannot. They arrange the features of a single dataset, whereas ours places the genes of several species on one grid, with orthologs at corresponding pixels. This is what enables shared pretraining across species: a model pretrained on human data alone reaches the highest zero-shot scores on both mouse datasets (0.445 on HypoMap and 0.599 on the mouse pancreas atlas, Figure 9). We do not claim that GW is the best possible layout for a single dataset. Our claim is that a layout that encodes gene relationships improves the representation over one that does not, and that the FGW layout is the one that extends to several species on a shared grid.

GenoMap arranges genes radially. Its released implementation measures image distance by the difference in pixel radius, so related genes lie at similar distances from the image center and form concentric rings, and two genes on opposite sides of the image can count as neighbors. This geometry suits two-dimensional visualization of expression. Our framework needs locality: a masked CNN predicts a gene from the pixels around it, so adjacent pixels must hold co-expressed genes, and the TransMap layout uses Euclidean pixel distances for this reason (Section 2).

We have revised the Related Work (Section 2) to explain how the TransMap layout differs from DeepInsight, IGTD and GenoMap in these two respects: locality for masked convolutional modeling, and a shared grid across species.

AuthorsOpenReview ↗

W2 + Q3: flatten readout, layout contribution and component ablations

W2. The contribution of spatial layout to the final cell representation lacks sufficient validation. The model flattens the deepest feature maps and linearly projects them into a 512-dimensional cell representation. Does this flattening-and-projection step discard some of the spatial information encoded in the image? From the perspective, what advantage does this approach offer over directly processing the same gene-expression values in a fixed random order?

Q3. How do individual model components contribute to the quality of the final cell representation? Could the authors provide ablations of the readout mechanism and loss components to clarify the respective contributions of different parts of designs?

Thank you for these very good questions, which go to the heart of the design. The architecture matters a great deal to us, so we ran a series of new experiments, all of them full pretraining runs, and we added the motivation for the flatten readout to Section 3.2, which the submission did not clearly state.

The flatten readout is the step that keeps the spatial information. Flattening is a reshape, and the linear projection that follows has a separate weight for every position and channel of the deepest feature map, so the cell representation depends on where each feature lies. We chose it because every pixel of an expression image is a fixed gene: a shifted image is a different cell, so the representation is not translation invariant. The early convolutional networks for natural images read out their features in the same way, and that layer was their heaviest part (62% of the parameters of AlexNet, 74% of VGG-16). Global average pooling (Lin et al., 2014) later replaced it, because the class of a natural image does not depend on where the object lies. Expression images do not have this invariance, and pooling over positions costs 0.21 in biological conservation (attention pooling in the table below).

The advantage over a fixed random order arises before the readout, in the 24 convolutional blocks of the encoder, whose filters see neighboring pixels. We tested the reviewer's alternative at the scale of the final model: a fixed random permutation of the pixels keeps every gene's value and the pixels shared by orthologs and destroys only the neighborhoods. Three such models, pretrained with the corpus, architecture, readout and schedule of the main run, reach 0.620 ± 0.022 in biological conservation and 0.441 ± 0.014 overall, against 0.700 and 0.490 for the GW layout, which is higher on each of the five datasets. An MLP that processes the gene vector directly, with the same corpus, masked genes, objective and a matched number of parameters, reaches 0.648 and 0.455.

The ablations of the loss components and the readout are in Table A5 of the revised manuscript:

ModelReadout parametersBioBatchOverall
Main model: both losses, flatten readout84.9M0.7000.1740.490
CLS loss alone84.9M0.6530.1850.466
Reconstruction loss alone–0.5960.2230.447
Readout: attention pooling1.5M0.4910.2170.382
Readout: 1×1 convolution to 128 channels, then flatten21.3M0.6890.1800.486
Readout: 1×1 convolution to 32 channels, then flatten5.3M0.6590.1890.471

(Means over the five datasets, scored as in Table 2. Under the reconstruction loss alone the readout is not trained, and the cell representation is the first 512 principal components of the flattened deepest feature map.)

The main model has the highest biological-conservation and overall scores. The two losses complement each other: removing the CLS loss costs 0.104 in biological conservation and removing the reconstruction loss 0.047. In the readout, what counts is that the weights are tied to positions. The flatten readout is the heaviest part of the model and has the highest scores, and it can be made lighter while staying position-specific: a 1×1 convolution shared by all positions reduces the deepest feature map to 128 channels before it is flattened, which removes 75% of the readout parameters at an overall score of 0.486 against 0.490.

Section 3.2 gives the motivation for the readout, Appendix G describes these models and the random-layout control (Figure 8), and Table A5 and Figure 9 in Appendix H.5 give the scores.

AuthorsOpenReview ↗

W1 + Q1 (1/2): grid versus gene graph (new run: graph model at the scale of the main model)

W1. The advantages of 2D embedding over directly utilizing gene relationships have not been fully established. Embedding gene relationships into a 2D grid can adapt to convolutional architectures, but it also introduces distortions in distance and neighborhood structure. While the expression values of selected genes are preserved, the relationship structure can only be approximated. Table A1 has evaluated layout fidelity, but the relationship between these metrics and representation quality is not fully explained, and there is a lack of model comparisons that directly utilize the same gene relationship graph.

Q1. Why choose a two-dimensional grid to represent gene relationships? What advantages are expected over learning directly on a gene relationship graph?

Response:

Thank you for this question on the central design choice of the paper. We took it as the occasion to train the model the reviewer asks for, one that uses the same gene relationship graph directly, at the scale of the main model.

The graph model is built on the graph that we construct for the layout. Each gene receives messages from its 24 nearest genes under the refined distances from which the layout is computed (Appendix E), so the model sees exactly the structure that the grid approximates, and it is given the human–mouse ortholog links explicitly. Corpus, masked gene sets, loss terms and schedule are those of the main model, and the size is matched (148.0M parameters against 147.1M).

ModelBioBatchOverall
TransMap: CNN on the GW layout0.7000.1740.490
Message passing on the gene graph0.6870.1950.490
Masked MLP on the gene vector0.6480.1650.455
CNN on random layouts (mean of three)0.6200.1710.441

(Means over the five datasets, scored as in Table 2.)

Even with the exact graph and explicit ortholog links, the graph model scores no higher than the convolutional model, which receives both only through the positions of the genes, and both stand clearly above the MLP and the random layouts. Two things follow. The framework around the encoder, namely the gene relationships we construct, the masking of gene modules and the two loss terms, is effective beyond the convolutional encoder. And there is more than one way to bring this inductive bias into a model: message passing is a good one, and the approximation the reviewer points to costs the grid nothing against it.

The grid turns gene relationships into locality, the property a convolutional encoder exploits, and we chose it for three reasons. The first is that it is friendlier to multiple species. To share parameters between species, a graph model has to be told which genes correspond. We merged every connected set of ortholog links into a group that shares one gene embedding and one set of readout weights: 171 groups hold 3 to 13 genes from one-to-many and many-to-many relationships, and 5,370 genes have no ortholog. For distant species ortholog calls become sparse and uncertain, and a model built on explicit correspondences depends on this curated knowledge. The FGW layout uses orthology only as a soft preference. Ortholog edges enter one cost term (α = 0.3) next to each species' own structure, one-to-many and many-to-many relationships are kept with attenuated weights (Equation 15), and genes without an ortholog are placed by the structure of their own species (Appendix C). The image also tolerates imprecise placement: half of the human–mouse ortholog pairs share a pixel and a third lie more than eight pixels apart, and a model pretrained on human data alone still reaches 0.445 on HypoMap and 0.599 on the mouse pancreas atlas, above UCE (0.436 and 0.550).

The second reason is that an image can be looked at. Conserved modules occupy corresponding regions in the human and mouse layouts (Figure 3b) and in those of five species from human to nematode (Figure 6).

The third is room to grow. A grid has a hierarchy: after downsampling, 18 of the 24 blocks of the main model run on 324 positions with 512 channels, whereas a graph model computes every layer on all 18,189 genes. Deep message passing is known to oversmooth (Li et al., 2018), and the usual remedy, graph coarsening, rebuilds such a hierarchy on the graph. Image encoders also come with established recipes for growing depth and width together (Liu et al., 2022), and in Table 1 the error of the GW layout falls from 458K to 15M parameters on every dataset.

TransMap is a new approach to representation learning for single cells, and we welcome that the gene relationships it builds on work with a second architecture as well. Section 4.3 states this result, Appendix G describes the graph model, and Figure 9 gives its scores. Our answer on Table A1 follows in the next comment.

Reference

Li et al. Deeper insights into graph convolutional networks for semi-supervised learning. AAAI, 2018.

Reviewer_jN7SOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The authors proposed TransMap, a framework that transforms single-cell gene expression profiles into two-dimensional images by arranging genes based on their relational structures. The method utilizes Gromov–Wasserstein optimal transport to preserve gene relationships within species and fused Gromov–Wasserstein transport to incorporate ortholog information across species. Subsequently, a masked self-supervised learning framework with a shared convolutional encoder is employed to learn cell representations from these images. The model was pretrained on approximately 732K human and mouse metacells and evaluated on multiple single-cell datasets. The proposed gene arrangements demonstrated improved masked reconstruction compared with random arrangements, while the pretrained model achieved competitive performance against existing foundation models on selected benchmarks. However, self-supervised fine-tuning decreased overall performance on three of the five benchmark datasets, and the results do not consistently demonstrate advantages over simpler methods such as PCA.

Critical strengths
  1. The authors proposed an interesting approach to represent single-cell gene expression as images while incorporating biological relationships among genes. The use of Gromov-Wasserstein transport and ortholog information provides a structured way to organize genes across species, which could be useful for developing image-based single-cell foundation models.

  2. The proposed gene arrangement was evaluated across five datasets and two model sizes, demonstrating lower masked reconstruction errors than random gene arrangements. Furthermore, the model was pretrained on approximately 732K human and mouse metacells and evaluated on five independent cell-embedding benchmarks. These experiments provide preliminary evidence for the feasibility of the proposed approach across datasets and species.

Critical weaknesses
  1. Although the authors evaluated the effect of self-supervised fine-tuning, the overall performance decreased on three of the five benchmark datasets. Furthermore, the fine-tuned TransMap model performs below PCA on most datasets, despite PCA requiring neither large-scale pretraining nor explicit batch integration. These results raise questions about the benefits of pretraining on approximately 732K metacells. The authors also reported decreased batch-correction performance across all datasets after fine-tuning. Additional experiments comparing pretrained and randomly initialized models, along with an investigation of the fine-tuning performance degradation, would be helpful in establishing the effectiveness and adaptability of the proposed framework.

  2. The primary motivation of the manuscript is to develop a unified representation for cross-species single-cell learning. However, the model was trained and evaluated only on human and mouse data, while the remaining species were considered only for gene-layout visualization. Additional experiments on cross-species cell-type alignment or knowledge transfer would be helpful in establishing the effectiveness of the proposed framework.

  3. Although the proposed model employs self-supervised learning, the pretraining metacells were constructed by aggregating cells within annotated coarse cell lineages. This introduces prior cell-type information during data preparation. It would be interesting to evaluate the model using metacells constructed without cell-type annotations or directly using individual cells to determine how much the learned representations depend on this preprocessing strategy.

  4. Although the authors compared TransMapVI with scVI in the supplementary experiments, the comparison does not directly evaluate the main pretrained TransMap framework. Furthermore, although the two models have similar parameter counts, both the input representation and model architecture differ between them. Without a TransMapVI variant using a random gene layout, the gains cannot be attributed to the layout rather than to the architecture. A vector-input counterpart trained on the same 732K metacells with the same objective and model size would help establish whether the image representation provides a real advantage.

Questions for authors

Please refer to weaknesses.

7 comments
AuthorsOpenReview ↗

W3: metacell construction

Although the proposed model employs self-supervised learning, the pretraining metacells were constructed by aggregating cells within annotated coarse cell lineages. This introduces prior cell-type information during data preparation. It would be interesting to evaluate the model using metacells constructed without cell-type annotations or directly using individual cells to determine how much the learned representations depend on this preprocessing strategy.

Response:

We thank the reviewer for this question. We would like to clarify what the lineage labels contribute to our data preparation pipeline and how we used them.

A metacell sums the raw counts of about 15 expression-similar cells from one sample and one coarse lineage. The label serves as a boundary for this aggregation: it decides which cells may be summed, and the grouping inside the boundary uses expression alone. The model receives expression profiles only, and its loss consists of the two self-supervised terms. The labels belong to the pretraining collections. The five benchmarks are disjoint from the pretraining corpus, are embedded as individual cells without metacell construction, and their labels are used only for scoring. The paragraph on metacell construction in Appendix F now describes this.

The reason we use the labels is data quality. Without the boundary, a metacell can pool cells of different lineages and produce an artificial doublet, for example a profile that is positive for both CD3E and CD79A. Removing such profiles is standard practice in single-cell quality control (Heumos et al., 2023), and the boundary keeps them out of the corpus.

The boundary and the benchmark operate at different resolutions. A pretraining unit has a median of 7 lineages, such as T cell, B cell, neuron or fibroblast, whereas the benchmarks are scored on 14 to 57 fine-grained classes per dataset (Appendix H.1). In HypoMap, for example, neurons are 57% of the cells and are scored as 16 classes, all of which fall in one coarse lineage. Separating coarse lineages is an easy task, and fine-grained types are much harder. On the same mouse brain dataset, all classifiers reach a median F1 above 0.99 for three classes and all of them drop for 92 classes (Abdelaal et al., 2019). Cross-species label transfer falls from 92%, 89% and 80% on broad labels to 73%, 67% and 49% on fine labels (Schächter et al., 2025), and recent methods reach 44% to 75% accuracy on 22 immune subsets (Heimberg et al., 2025). The fine-grained distinctions that the benchmark scores lie within coarse lineages, beyond what the boundary encodes.

Like the reviewer, we were curious whether individual cells would serve better, and we ran the second of the two evaluations the reviewer suggests. Starting from the pretrained model, we continued pretraining on the 11.2 million individual cells from which the training metacells were built. The two models see the same cells, and the continued one uses neither aggregation nor lineage labels (Appendix G). The continued model receives this training in addition to the pretraining on metacells, so its scores would rise if individual cells were the better training data. Instead, the scores decreased: overall 0.424 versus 0.490, biological conservation 0.604 versus 0.700, and batch correction 0.154 versus 0.174 (new Table A6 and the paragraph "Metacells versus individual cells" in Appendix H.5). We attribute this to the sparsity and noise of single-cell profiles, which supports pretraining on metacells.

References

  • Abdelaal et al. A comparison of automatic cell identification methods for single-cell RNA sequencing data. Genome Biology 20:194, 2019.
  • Heimberg et al. A cell atlas foundation model for scalable search of similar human cells. Nature 638:1085-1094, 2025.
  • Schächter et al. scSpecies: enhancement of network architecture alignment in comparative single-cell studies. Genome Biology 26:397, 2025.
  • Heumos et al. Best practices for single-cell analysis across modalities. Nature Reviews Genetics 24:550-572, 2023.
AuthorsOpenReview ↗

W4: matched vector-input control at the scale of the main model

W4. Although the authors compared TransMapVI with scVI in the supplementary experiments, the comparison does not directly evaluate the main pretrained TransMap framework. Furthermore, although the two models have similar parameter counts, both the input representation and model architecture differ between them. Without a TransMapVI variant using a random gene layout, the gains cannot be attributed to the layout rather than to the architecture. A vector-input counterpart trained on the same 732K metacells with the same objective and model size would help establish whether the image representation provides a real advantage.

Response:

We trained the vector-input counterpart the reviewer asks for, and we pretrained the main framework on random gene layouts as well. Both controls are at the scale of the main pretrained TransMap model, and together they attribute its advantage to the image representation and, within it, to the gene relationships encoded in the layout.

The vector-input model is an MLP over the gene vector trained on the same 732K metacells with the same masked genes, the same two loss terms, the same optimizer and number of steps, and a matched size (149.4M parameters against 147.1M). It receives the expression values and the visibility indicators of the genes, the two channels of the expression image without their arrangement, and its masks are the patch masks of the GW layout mapped to gene indices. The random-layout models keep the convolutional architecture and change only the arrangement: one random permutation of the pixels, applied to the human and the mouse layout alike, so that every gene keeps a pixel of its own and only the neighborhoods are destroyed. Three such models were pretrained with different permutations under the settings of the main run.

ModelBioBatchOverallDKDGTEx v9ImmunePancreasHypoMap
TransMap (images, GW layout)0.7000.1740.4900.5160.4190.5240.5710.419
Vector input (MLP, same metacells, masks and objective)0.6480.1650.4550.4720.4080.5170.5090.369
Images, random layouts (mean of three models)0.6200.1710.4410.4500.3730.4800.5500.350

(Scored as in Table 2: means over the five datasets on the left, the overall score of each dataset on the right. The random-layout row is the mean of three models; each other row is one pretraining run.)

TransMap scores higher than the vector-input model on every dataset in overall score and on four of the five in biological conservation, 0.700 against 0.648 on average, and higher than the random layouts in biological conservation on every dataset, 0.700 against 0.620 ± 0.022. The two controls answer the two halves of the question. Replacing the images by vectors under the same objective lowers the scores, so the image representation provides the advantage. Replacing the GW layout by a random one lowers them to the same level, so the advantage does not come from the convolutional architecture by itself but from the gene relationships that the layout places next to each other. The TransMapVI comparison in the appendix is the single-species precursor of these experiments; the attribution now rests on the main model.

Appendix G describes both controls (Random-layout control, Masked MLP ablation), Figure 8 gives the random-layout loss curves and scores, Figure 9 the scores of every model on every dataset and metric, and Section 4.3 summarizes both results.

I appreciate the authors' efforts to address my concerns, particularly through the new ablation studies and clarification of metacell construction. These results strengthen the paper. However, my concerns regarding fine-tuning performance and cross-species validation remain. Therefore, I will maintain my original score of 2.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of the revision and of the new experiments

We sincerely thank the Area Chairs and all four reviewers (jN7S, cDs7, mf9K, CLUf) for their constructive feedback, which substantially strengthened our work.

We took every request for an experiment seriously and treated the discussion as an opportunity to make the work as solid as we could. During the discussion period we trained about twenty additional versions of the full 147M-parameter model, each a complete pretraining run on the 732K-metacell corpus: random-layout controls, a vector-input MLP, a graph neural network on the gene graph, single-species models, readout and loss ablations, a model trained on individual cells, and repeated runs with new random seeds. We added about fifty fine-tuning and adaptation runs, re-embedded and re-scored every method of the benchmark under one protocol, and ran two new cross-species experiments covering seven species. The results are now in the revised manuscript, and we are grateful to the reviewers for the questions that prompted them. Some of these results were posted after the follow-up comments of Reviewers jN7S and CLUf, namely our replies to jN7S W1 and W2 and the cross-species experiments of Appendix I.

The reviewers' questions center on five core concerns, which the new experiments address directly.

Does the gene layout improve the final model, and not only small ones? Yes. With everything else fixed, replacing the layout by a random one lowers biological conservation from 0.675 ± 0.016 (five runs) to 0.620 ± 0.022 (three runs), and even the weakest layout run scores above the strongest random one. A size-matched MLP trained on the same gene vector and the same masked genes stays at the level of the random layouts. A graph network given the exact gene graph and explicit ortholog links reaches the same score as our released model (0.490 overall), so the two-dimensional approximation costs nothing. The gain is not an effect of masking either: masking is harder under our layout, and embeddings are computed without masks. (CLUf Q2; mf9K W1, W4, Q1–Q3; cDs7 W1, W2, Q1, Q2; jN7S W4)

Does the model share knowledge across species? Yes. A model pretrained on human data alone, which has never seen a mouse cell, has the highest zero-shot score on both mouse benchmarks, above UCE, which was pretrained on mouse data. Human and mouse pancreas cells of the same type align on par with UCE (0.759 for both), with twice its recall of the rare gamma cells. Without any training, the model separates the cell types of five species it has never seen, from rat to fruit fly and nematode, with 79–88% nearest-neighbor accuracy. Joint pretraining also improves biological conservation over either species alone. We did not pretrain on layouts built independently per species: the same pixel would then hold unrelated genes in human and mouse, so the encoder could share nothing between them. The human to mouse transfer already measures what the alignment contributes. (CLUf W1, Q1; mf9K W2, Q4; jN7S W2).

Is the benchmark comparison fair? It is now fully matched: every method is embedded and scored by us on the same cells, labels and metrics, all foundation models receive the full gene matrix, biological conservation and batch correction are reported separately, and TransMap is reported over five pretraining runs. TransMap matches UCE, the only other zero-shot model that covers both species, at a quarter of its size, on benchmark studies that are excluded from its corpus. (mf9K W3; cDs7 W3, Q4)

Why does fine-tuning lower the score? Label-free fine-tuning keeps biological conservation and loses only batch correction, because masked prediction on a single dataset rewards batch differences. Pretraining clearly helps: the same model trained from scratch on each dataset scores lower. Our batch-aware adaptation addresses the cause and scores above PCA on all five datasets and above every zero-shot model on two. (cDs7 W3a; jN7S W1)

Is each design choice justified? Each was tested by a full pretraining run. The position-specific readout keeps the information that pooling destroys (pooling costs 0.21 in biological conservation) and can be made 75% lighter at the same score; both loss terms contribute; and training on individual cells instead of metacells lowers the scores. (cDs7 W2, Q3; mf9K Q5; jN7S W3)

We also aligned the layout equations with the implementation, documented the source of every pretraining collection and the exclusion of benchmark studies, redrew Figure 4 with UCE and added plots colored by cell type and by batch, and condensed the Table 2 caption. (CLUf W2; mf9K Q6; cDs7 W3b, W3d)

Together, these experiments establish that a gene layout built from gene relationships and orthology is an effective input for single-cell representation learning: its benefit holds at full scale and across repeated runs, it carries knowledge across species, and the first model built on it already reaches the level of established foundation models.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.