acceptodds
Under review as a conference paper at ICLR 2027

Language as the Interface: Foundation-Model Contrastive Learning Links Transcriptomes and Electrophysiology

Abstract

Integrating transcriptomic and electrophysiological data is essential for building multimodal foundation models for neuroscience. Patch-seq provides paired measurements of gene expression and intrinsic electrophysiology from the same neuron, establishing a basis for training cross-modal models. Here we introduce LangPatch, a foundation-model-based contrastive learning framework that uses paired Patch-seq data to align pretrained GenePT representations with electrophysiological phenotypes through a language-based interface. Gene descriptions and verbalized electrophysiological profiles are embedded by the same frozen text encoder. A context adapter and projection modules connect the modalities through paired contrastive learning. Across mouse visual, mouse motor, and human cortical cohorts, LangPatch achieves the highest mean transcriptome-to-electrophysiology prediction correlation among the evaluated foundation-model and representation-learning methods. It also improves held-out cross-modal alignment in the two mouse cohorts (FOSCTTM 0.107/0.135 vs. 0.208/0.222 for JAMIE, an existing cross-modal Patch-seq imputation method). It predicts transcriptomic family, type, cortical layer, and marker-gene expression from electrophysiology, exceeding other baselines on most endpoints. More importantly, the method transfers across brain areas and species: a model trained on mouse visual cortex predicts electrophysiology in motor cortex with approximately 70% correlation retention and in human cortex with 47% (58% on acute-slice recordings). Together, these results demonstrate alignment between molecular and functional representations of neurons, providing a building block for multimodal foundation models in neuroscience.

Primary area:
applications to neuroscience & cognitive science
Keywords:
foundation models, multimodal alignment, contrastive learning, text embeddings, cross-modal prediction, zero-shot transfer, computational neuroscience, transcriptomics, electrophysiology

Reviews

Among:
Shape:
1234this paper, Top 4%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.33 over 3 reviews, above 96% of the others.
Reviewer_wFFJOpenReview ↗
Rating
4(+1)
Confidence
3
Informativeness
3
Presentation
3
Soundness
3(+1)
Read review
Paper summary

This is a strong and well organized work that uses text embeddings as a shared space for gene expression and electrophysical neuron data. The authors use CLIP to align the two embeddings and translate between modalities. I find this work easy to follow and the method to be interesting and novel, despite performance improvements being occasionally only modest over previous methods. Transfer between mouse cortices is effective in several electrophysical features, while cross-species transfer is not as strong. Attribution/interprability results are mostly anecdotal but are presented as such.

Critical strengths

The proposed method provides state of the art performance translating between gene expression and neuron electrophysical data. The gains are modest in some circumstances, but the method consistently outperforms the competing methods and beats the best competitor - JAMIE - the majority of the time. The method also transfers somewhat between mouse cortices, showing some promise for general zero shot transfer.

Using a pretrained text encoder as a shared embedder for disparate modalities in this way is a novel idea and could be broadly useful. This allows modalities to begin in the same latent space without any (or with minimal) training. Vocabularies/fields in certain data can be missing without causing the resulting representation to break and new modalities could be added easily in theory. Although there might be some information lost in the text bottleneck, I could see a similar method being effective applied to different modalities and domains.

Critical weaknesses

Ablations are lacking cases that prove the full method is necessary. While we see improvements with the proposed method, it is not clear whether the shared text embedding specifically is useful. Would CLIP embeddings of separate numerical expression and numerical electrophysical embeddings work as well? Or genePT text + numerical electrophysical?

Cross species transfer is somewhat weak. Mouse->human transfer gives poor performance, but does improve significantly when fine tuning on only a few samples. Would human->mouse transfer work?

Questions for authors

Please clarify whether the gene and electro-physical text embeddings use the same model. Fig 1 and the overview paragraph say they 'use the same frozen general purpose text embedder' (line 191) but shortly after (line 201) says that the gene text is passed through frozen GenePT embedding, which is what I originally assumed to be the case.

It is important to at least quantify the contribution of the choice to use the text space (see first weakness above). Would it be possible to test versions with choices of text/numerical embeddings for both expression and electrophyscial modalities?

I think it could be good to highlight the mouse->human performance improving with some extra low data training. This is up to the authors though.

3 comments

Thank you for the careful reading and the constructive questions. We have run all the experiments you suggested.

1. Is the shared text space useful? Text or numerical embeddings for each modality (Weakness 1, Question 2)

We evaluated all four combinations of text-based and numerical embeddings for the two modalities. LangPatch, which uses text for both, achieves the best forward-prediction accuracy.

(a) Text or numerical, for each modality. Forward prediction on held-out cells (Metrics: Pearson r / raw MAE / AUROC, using five-fold means). AUROC is computed per feature: held-out cells are split at the feature's median, and the prediction is scored on how well it separates the upper half from the lower half (0.5 is chance).

Expression embeddingElectrophysiology embeddingVisual cortexMotor cortex
LangPatchtext (GenePT + adapter)text (verbalized record)0.521 / 9.09 / 0.7770.532 / 7.90 / 0.807
New ablationnumerical (MLP)numerical (MLP)0.419 / 10.61 / 0.7170.495 / 8.66 / 0.782
New ablationtext (plain GenePT, no adapter)numerical (MLP)0.507 / 9.24 / 0.7670.495 / 8.17 / 0.783
New ablationnumerical (MLP)text (verbalized record)0.472 / 9.62 / 0.7540.503 / 8.53 / 0.785
New ablationnumerical (linear)text (verbalized record)0.501 / 9.18 / 0.7710.517 / 8.12 / 0.786
Original Table 1, GenePT–ephys VAEtext (plain GenePT, no adapter)numerical (VAE)0.455 / 9.88 / 0.7340.021 / 10.72 / 0.513
Original Table 1, JAMIEnumerical (autoencoder)numerical (autoencoder)0.453 / 10.05 / 0.7390.432 / 8.91 / 0.757

LangPatch achieves the best performance on all three metrics in both cohorts. Replacing both modalities with numerical embeddings reduces r by 0.10 on visual cortex and by 0.04 on motor cortex. Pretrained numerical expression embeddings (MLPs on frozen Geneformer, UCE and Nicheformer; Table 1) achieve at most r = 0.492 and 0.503.

All new variants use the same training objective, folds and readout head as the LangPatch row, so that only the embeddings differ. The numerical encoders take log-normalized expression and quintile-binned electrophysiology features as input. The last two rows are taken from Table 1 of the paper.

(b) Choice of text embedder. We re-embedded the gene descriptions and recordings with ten alternative embedders and retrained LangPatch with all other settings fixed. FOSCTTM measures the alignment error on held-out cells (0 is perfect, 0.5 is chance).

Text embedder (used for both modalities)FOSCTTM ↓ visualFOSCTTM ↓ motorForward MAE ↓ visualForward MAE ↓ motor
TF-IDF + SVD (bag of words, no language model)0.2250.2719.057.77
bge-base-en-v1.5 (open, 110M)0.1890.1939.047.70
all-MiniLM-L6-v2 (open, 22M)0.1750.2039.097.78
bge-large-en-v1.5 (open, 335M)0.1730.1699.027.73
text-embedding-ada-002 (API)0.1400.1599.067.69
bge-m3 (open, 568M)0.1390.1679.027.61
Qwen3-Embedding-8B (open)0.1240.1459.067.64
Cohere embed-5-pro (API)0.1210.1419.037.69
text-embedding-3-small (API)0.1160.1389.027.72
Qwen3-Embedding-4B (open)0.1150.1399.017.70
text-embedding-3-large (API; LangPatch)0.1090.1398.987.65

FOSCTTM decreases from 0.225 and 0.271 with a bag-of-words representation to 0.109 and 0.139 with the embedder used in LangPatch, indicating that alignment quality improves with the quality of the text embedder.

(c) Zero-shot generalization to another laboratory's feature set. You note among the strengths that "Vocabularies/fields in certain data can be missing without causing the resulting representation to break". We tested the extreme case in which the entire feature set differs. The visual-cortex model, trained on 39 Allen Institute features, was applied to motor-cortex recordings described by the 29 features of Scala et al. (https://doi.org/10.1038/s41586-020-2907-3; no feature names in common), without a feature correspondence and without retraining. Family label-transfer accuracy between the recording and transcriptome embeddings of the same cells is 0.51 against a permutation null of 0.19, and 0.78 against 0.30 on interneuron families (the classes seen during training). This setting is infeasible for numerical encoders: their input layer is tied to the feature set of the training cohort, so applying them to another laboratory's features requires a manually constructed feature correspondence or retraining.

2. Would human → mouse transfer work? (Weakness 2)

Yes, and human → mouse transfer is stronger than mouse → human. We trained LangPatch on the 612 human neurons only and applied it without adaptation to the 3,654 mouse visual-cortex neurons, following the evaluation protocol of the paper.

DirectionTraining cellsMean rConcepts above the nullRetentionGene-permuted control (mean r)
Mouse → human (paper)3,654 mouse0.2818 of 2547% (58% on acute slices)0.00
Human → mouse (new)612 human0.3923 of 2463%−0.02

Input resistance, rheobase, membrane time constant and after-hyperpolarization transfer with r between 0.62 and 0.68. One of the 25 concepts (the adaptation index) has no counterpart among the human features, so 24 concepts are evaluated in this direction.

Zero-shot transfer therefore holds across species in both directions, and is stronger toward the species with cleaner target recordings despite six times fewer training cells. This may suggest that mouse → human transfer is limited more by variability in the human recordings than by the species gap.

3. Do the gene and electrophysiology text embeddings use the same model? (Question 1)

Yes. Both use OpenAI text-embedding-3-large (3,072 dimensions). GenePT is not a separate network. It passes each gene's text summary through a general-purpose text embedder, the newest version via text-embedding-3-large. We embedded the recordings with that same model. As a check, we re-embedded the gene summaries ourselves and recovered the released GenePT embeddings (cosine similarity 1.000).

The sentence you point to (line 201 of the submission, in Sec. 3.1) now reads: "Gene j is represented by g_j = E(description of gene j), computed with the same embedder E as the recordings. These vectors are the released GenePT embeddings (Chen & Zou, 2025), which are text-embedding-3-large applied to each gene's NCBI and UniProt summary." The caption of Fig. 1 has been updated accordingly.

4. Highlighting few-shot mouse → human transfer (Question 3)

We have done so: Sec. 5.3 now devotes a paragraph to this result. Calibrating the mouse-trained model on 20 paired human cells yields an error within 2.5% of that obtained with 100 cells. With 25 labelled human cells, a ridge head on the frozen mouse-trained representation achieves r = 0.270, compared with 0.197 for Plain GenePT and 0.195 for Geneformer.

Changes in the revised PDF. Sec. 5.4 with Table 2 and Fig. 4 (new): text versus numerical embeddings, text-embedder ablation and cross-laboratory test. Sec. 5.3: cross-laboratory test, human → mouse transfer and the few-shot paragraph. Sec. 3 and the caption of Fig. 1: clarification of the embedder. Appendix A (new): text-embedder table (Table 4) and per-concept human → mouse results (Table 5).

Thank you for performing the requested experiments and answering my questions. I believe that these new results, particularly the text/numerical comparison under the same scheme as the proposed method and the new missing feature exploration greatly strengthen this work. I additionally appreciate the clarification on the text embedder (text-embedding-3-large) - I think it is important to make this part clear and the updated language you show does this well. I have raised my score to a 4.

Reviewer_gm6oOpenReview ↗
Rating
3(+2)
Confidence
2
Informativeness
2
Presentation
1
Soundness
3(+1)
Read review
Paper summary

This paper introduces LangPatch, a method that can predict seq-patch ephys. features from transcriptomic features and vice versa. The core method builds on top of GenePT by adding a few model components (normalizations and adapters), and a CLIP-like pretraining strategy that aligns genetic and ephys features. A novel approach is to "verbalize" the ephys features into text and embed them via text-embedding models.

The resulting representations are able to predict ephys features from transcriptomic features, and vice versa. Additionally, the paper also studies model performance when the model is trained and evaluated on different brain regions and species.

Critical strengths
  • Study of model performance when the model is trained and evaluated on different brain regions and species is interesting to see.
Critical weaknesses

W1. Writing: Many portions of this paper are quite hard to read. Section 5, which attempts to describe the paper's main experimental results, is especially hard to parse. It contains run-on sentences that are dense with numbers, which seem to be stating the results instead of contextualizing it with the author's understanding. Another writing-related issue is that the paper keeps reporting performance numbers as "0.107/0.135 vs. 0.208/0.222" throughout the paper (this particular example is taken from the abstract!). But this A/B notation (which this paper uses to mean performance on visual/motor cohorts) is introduced on page 7. So, for more than half of the paper, the reader has no understanding of what these reported numbers exactly mean.

W2. Lack of ablations: This method adds quite a few components on top of GenePT. (1) Context adapter on the gene side, (2) the extra cosine-penalty loss to keep context-adapter's output close to GenePT embeddings, (3) cosine-penalty that keeps h_i close to unadapted cell embeddings, and (4) auxiliary regression of y from both ephys and gene sides. However, none of these are ablated. We, thus, don't know what the contribution of these techniques is, or if all of these (and their associated hyperparameters) are necessary for LangPatch's performance. Figure 2c shows that the training time of LangPatch is quite small (~60s), so it should be to do rigorous ablations.

Questions for authors

I request authors to address both W1 and W2 in the rebuttal period.

Also in line 492: "Every number reported in the paper is regenerated by the released scripts from the released result files (CSV)." However, there are no "released result files" attached to this submission.

4 comments

Thank you for the direct and specific review. We have rewritten Section 5, removed the paired notation, released the result files, and run the requested ablations. Beyond the four components, the ablations also cover the two central design choices of the method: the language interface and the choice of language model.

W1. Writing

Section 5 has been rewritten. Each subsection now opens with a one-sentence statement of the finding, followed by its interpretation and then the supporting numbers. Standard deviations, p-values and the per-feature comparison with every baseline are now in the tables (Table 1 and the new Table 8). For example, the opening of Sec. 5.1, previously a single sentence containing all cohort means, standard deviations, feature counts and p-values, now reads:

"Among the methods compared, LangPatch predicts electrophysiology from the transcriptome most accurately on all three cohorts (Table 1). It has the highest correlation, the highest AUROC and the lowest error in mouse visual cortex, in mouse motor cortex and in human cortex. Against JAMIE, an existing cross-modal Patch-seq imputation method, it is more accurate on 36 of 39 features in visual cortex and on 27 of 29 in motor cortex."

The "0.107/0.135" notation. We agree that this notation was used before it was defined. The abstract and introduction no longer contain paired numbers. The abstract now reads "FOSCTTM 0.107 in visual cortex and 0.135 in motor cortex, against 0.208 and 0.222 for JAMIE". In Sec. 5, the tables have one column per cohort, and the text names the cohort for each value.

W2. Ablations

We ablated the method at three levels: the four components you list and their hyper-parameters, the language interface, and the choice of language model. All runs use the same five folds and held-out cells as the paper. FOSCTTM measures the alignment error on held-out cells (0 is perfect, 0.5 is chance).

(a) Component ablation: each of the four components removed individually, and all four removed jointly. Forward prediction uses the same per-feature heads as the full method.

ModelVisual cortex r / raw MAEMotor cortex r / raw MAEFOSCTTM ↓ visualFOSCTTM ↓ motorFamily label transfer ↑ visualmotor
LangPatch (all four components), rerun with the ablations0.521 / 8.980.521 / 7.650.1090.1390.9270.822
without (1) the context adapter0.516 / 9.070.501 / 7.900.1100.1590.9210.745
without (2) the gene-cosine penalty0.519 / 9.040.522 / 7.680.1120.1350.9210.873
without (3) the cell-preserve penalty0.521 / 9.010.520 / 7.610.1080.1430.9230.809
without (4) the auxiliary regression, transcriptome side0.521 / 9.030.518 / 7.730.1100.1360.9260.833
without (4) the auxiliary regression, recording side and its warm-up0.522 / 9.010.526 / 7.680.1160.1430.9290.839
without (4) the auxiliary regression, both sides0.524 / 8.990.529 / 7.650.1140.1400.9270.850
without all four (plain contrastive learning on the frozen embeddings)0.519 / 9.040.506 / 7.920.1180.1700.9150.800
JAMIE (from the paper, for scale)0.453 / 10.050.432 / 8.910.2080.2220.7770.551
  • The context adapter (1) contributes most. Removing it degrades prediction on 31 of 39 visual-cortex features and 23 of 29 motor-cortex features (p < 0.001 for both). On motor cortex, the alignment error increases from 0.139 to 0.159 and family label-transfer accuracy decreases from 0.822 to 0.745. On the smaller human cohort (612 cells), the alignment error increases from 0.283 to 0.312.

  • The gene-cosine penalty (2) improves prediction on visual cortex: removing it degrades prediction on 27 of 39 features (p = 0.005).

  • The auxiliary regression (4) on the recording side reduces the alignment error on visual cortex in every fold (0.109 vs. 0.116 without it).

  • The cell-preserve penalty (3) reduces the alignment error on motor cortex (0.139 vs. 0.143 without it).

  • All four components jointly improve over plain contrastive learning: removing them increases the alignment error in every fold of both cohorts (0.118 vs. 0.109 on visual cortex and 0.170 vs. 0.139 on motor cortex) and degrades prediction on 22 of 29 motor-cortex features (p = 5×10⁻⁴).

W2. Ablations (continued)

(b) Hyper-parameter sensitivity. The alignment stage has four hyper-parameters: the two penalty weights, the contrastive temperature and the adapter gate scale. We varied each one individually to a lower and a higher value, keeping the other three at their defaults.

Hyper-parameter (default)Lower valueFOSCTTM visualFOSCTTM motorHigher valueFOSCTTM visualFOSCTTM motor
Gene-cosine weight (0.5)0.250.1100.1341.00.1110.139
Cell-preserve weight (0.1)0.050.1110.1360.20.1080.137
Temperature (0.07)0.030.1170.1540.150.1020.131
Gate scale (0.1)0.050.1110.1400.20.1100.133

Alignment is stable across all eight settings: FOSCTTM remains between 0.102 and 0.117 on visual cortex and between 0.131 and 0.154 on motor cortex, compared with 0.208 and 0.222 for JAMIE.

(c) Language interface: text versus numerical encoders for each modality. All models in this table, including LangPatch, are evaluated with the same readout head, so that only the encoders differ.

Expression encoderElectrophysiology encoderVisual cortex r / raw MAEMotor cortex r / raw MAE
text (LangPatch)text (LangPatch)0.521 / 9.090.532 / 7.90
numerical (MLP)numerical (MLP)0.419 / 10.610.495 / 8.66
text (plain GenePT, no adapter)numerical (MLP)0.507 / 9.240.495 / 8.17
numerical (MLP)text0.472 / 9.620.503 / 8.53
numerical (linear)text0.501 / 9.180.517 / 8.12

LangPatch, which uses text for both modalities, achieves the best performance on both cohorts. Replacing both with numerical encoders reduces r by 0.10 on visual cortex and by 0.04 on motor cortex.

(d) Choice of language model: ten alternative text embedders, with all other settings fixed.

Text embedder (used for both modalities)FOSCTTM ↓ visualFOSCTTM ↓ motorForward MAE ↓ visualForward MAE ↓ motor
TF-IDF + SVD (bag of words, no language model)0.2250.2719.057.77
bge-base-en-v1.5 (open, 110M)0.1890.1939.047.70
all-MiniLM-L6-v2 (open, 22M)0.1750.2039.097.78
bge-large-en-v1.5 (open, 335M)0.1730.1699.027.73
text-embedding-ada-002 (API)0.1400.1599.067.69
bge-m3 (open, 568M)0.1390.1679.027.61
Qwen3-Embedding-8B (open)0.1240.1459.067.64
Cohere embed-5-pro (API)0.1210.1419.037.69
text-embedding-3-small (API)0.1160.1389.027.72
Qwen3-Embedding-4B (open)0.1150.1399.017.70
text-embedding-3-large (API; LangPatch)0.1090.1398.987.65

FOSCTTM decreases from 0.225 and 0.271 with a bag-of-words representation to 0.109 and 0.139 with the embedder used in LangPatch.

These tables are included in the new Sec. 5.4 (Table 2, Fig. 4) and Appendix A (Tables 3 and 4).

Novelty of the method

As you note in your summary, the novel step is to verbalize the recordings and embed them with a text-embedding model. This step is the core of the method: a single frozen general-purpose text embedder serves as the shared embedding space for two measured modalities, neither of which is text. A transcriptome is represented through the text descriptions of its genes, and a recording through a verbalized list of its measurements. No modality-specific encoder is trained from scratch: alignment requires only a 0.4M-parameter adapter and two linear projections, trained in about one minute on 600 to 3,700 cells. To our knowledge, this is the first such approach for single-cell data. GenePT and other language-based single-cell models relate a single modality to text, and JAMIE trains a separate encoder per modality on each cohort. Sec. 1, Sec. 2 and the overview of Sec. 3 now state this explicitly.

The ablations you requested show that this design is effective: replacing the text embeddings of both modalities with numerical encoders reduces r from 0.521 to 0.419 on visual cortex and from 0.532 to 0.495 on motor cortex (table (c) above), and removing the four added components increases the alignment error in every fold of both cohorts (table (a) above).

Novelty of the method (continued)

Alignment quality scales with the quality of the text embedder (table (d) above), so LangPatch benefits directly from stronger language models. JAMIE, which trains modality-specific encoders, has an alignment error of 0.208 on visual cortex, compared with 0.109 for LangPatch.

LangPatch outperforms models built on single-cell foundation models. Geneformer, UCE and Nicheformer are pretrained on tens of millions of cells, yet MLPs on their embeddings achieve r of at most 0.492 on visual cortex and 0.503 on motor cortex, compared with 0.522 and 0.520 for LangPatch (Table 1).

LangPatch generalizes zero-shot to another laboratory's feature set, which is infeasible for models with numerical encoders. The visual-cortex model was trained on recordings described by 39 Allen Institute features. We applied it to motor-cortex recordings described by the 29 features of another laboratory (https://doi.org/10.1038/s41586-020-2907-3; no feature names in common), without a feature correspondence and without retraining.

Zero-shot testCellsFamily label transferPermutation null
Visual-cortex model → motor-cortex recordings (29 features)1,2080.510.19
same, interneuron families (the classes the model was trained on)8170.780.30
Motor-cortex model → visual-cortex recordings (39 features)3,6540.650.31

Family label-transfer accuracy is more than twice the permutation null in every row. Models with numerical encoders are not applicable in this setting: the input layer of JAMIE, a VAE or an MLP is tied to the feature set of its training cohort, so applying it to another laboratory's features requires either a manually constructed feature correspondence or retraining on paired cells from that laboratory. LangPatch requires neither, because the new features are verbalized with their names and units and embedded by the same text embedder. Since laboratories extract different feature sets, this property determines whether a trained model can be reused across laboratories.

Zero-shot transfer holds across species in both directions. Since you found the cross-region and cross-species transfer interesting, we added the missing direction: LangPatch was trained on the 612 human neurons only and applied without adaptation to the 3,654 mouse visual-cortex neurons.

DirectionTraining cellsMean rConcepts above the nullRetentionGene-permuted control (mean r)
Mouse → human (paper)3,654 mouse0.2818 of 2547% (58% on acute slices)0.00
Human → mouse (new)612 human0.3923 of 2463%−0.02

Transfer is stronger in this direction than from mouse to human, despite six times fewer training cells. One of the 25 concepts has no counterpart among the human features, so 24 concepts are evaluated.

These results are reported in Sec. 5.3, Fig. 4c,d and the new Appendix A (Table 5).

Another benefit is that the framework is easy to extend to additional modalities: in principle, the same frozen text embedder and contrastive objective apply to any modality that can be verbalized as named quantities, such as morphological features.

Released result files (your note on line 492 of the submission)

Thank you for pointing this out. The anonymous repository contained the code and inputs but not the result files. It now includes a results/ folder with the CSV files underlying every table and figure of the paper and every table in this response, together with the script that regenerates each table.

We hope that the rewritten Section 5, the ablations and the released files address your concerns, and we would be grateful if you would reconsider your rating in light of them.

I thank the authors for sincerely engaging with my questions. With the added ablations and improved writing, my main concerns are addressed, and I am increasing my score to 3.

Just a suggestion: For future rebuttals that allow manuscript modifications, I recommend the authors to color code the changed text so that it is easier for the reviewers to identify which parts of the manuscript changed. This is just a friendly suggestion for future, and it has no bearing on my rating of this submission.

Reviewer_EBFFOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents LANGPATCH, a framework for linking transcriptomic profiles and electrophysiological measurements. It combines GenePT embeddings and text embeddings through a cohort-conditioned gene adapter and contrastive learning, while keeping the text encoder frozen. The authors evaluate the model on diverse tasks. The paper reports the highest forward prediction correlation among the evaluated representation methods and improved held-out alignment on two mouse cohorts.

Critical strengths

S1. The model is evaluated on diverse tasks, including forward prediction, reverse readout of cell identity, cross-modal alignment, zero-shot transfer across areas and species, and gene-level attribution. This breadth fits the paper’s aim of providing a building block for multimodal foundation models.

S2. The held-out cross-modal alignment results are promising. On the two mouse cohorts, FOSCTTM improves from 0.133/0.186 for the unadapted GenePT alignment control to 0.107/0.135 for the full method.

S3. The framework does not require fine-tuning of the foundation model’s weights. It operates on cached text embeddings, so it works even when the encoder is available only through an API.

Critical weaknesses

W1. The ML novelty is limited. All components are existing methods. This work uses GenePT, CLIP’s InfoNCE, and TabLLM-style serialization. The authors state that the combination is new. However, for an ML venue like ICLR, I don't see this combination as a strong enough methodological contribution. The combination would be more convincing if it clearly outperformed its components used alone.

W2. The benefit of verbalization is not tested. Each value in the record comes with its quintile label, so the embedding may rely mostly on this binned information. Without a comparison to an MLP encoder on quintile-binned features, the gain from verbalization cannot be assessed. Also, authors argue that verbalization lets cohorts with different feature vocabularies land in the same space. However, there is no experiment that tests it.

W3. The experimental settings could be more carefully matched across methods. The proposed method uses per-feature heads selected on the validation fold, while the baselines use one shared head with default settings. Given the small margin over Plain GenePT, the gain may come from the head rather than the representation. On the human cohort, where both methods keep their mouse configurations, the proposed method scores 0.400 against 0.406 for JAMIE

Questions for authors

Q1. Which component do the authors consider the main methodological contribution? How much does performance drop when only that component is removed?

Q2. How does the method compare with an MLP encoder on quintile-binned features, trained with the same contrastive objective?

Q3.How much of the alignment gain over JAMIE remains when the adapter uses only label-free statistics?

6 comments

Thank you for the detailed review and the precise questions. We have run all the comparisons you requested.

W1. Novelty

We candidly agree with your point that ML novelty was a limitation of our previous manuscript: rather than inventing a new standalone backbone or loss function from scratch, our primary focus is tackling a new scientific problem and expanding its boundary, bridging molecular and functional single-cell modalities by combining pretrained foundation models, feature verbalization, and contrastive alignment. Unlike prior methods that map a single modality to text (e.g., GenePT, CellWhisperer) or train cohort-specific encoders per modality (e.g., JAMIE, scCLIP), LangPatch aligns two non-textual modalities within one frozen language space using only a 0.4M-parameter adapter and linear projections (~1 minute of training), outperforming single-cell foundation models pretrained on tens of millions of cells (Table 1).

Crucially, we strongly agree with you that when combining existing building blocks, demonstrating that their combination brings qualitatively new changes and clearly surpasses individual components is essential, and directly answers the question of novelty. Guided by this insight, we conducted extensive experiments to quantify the contribution of each component and, in particular, to explore scalable and inspiring questions enabled by this synthesis, such as the necessity of shared language representations over numerical encoders, as detailed below.

Alignment quality scales with the quality of the text embedder. With all other settings fixed and only the embedder replaced (W2 (c)), the alignment error on visual cortex (FOSCTTM; 0 is perfect, 0.5 is chance) is 0.225 without a language model (bag-of-words), 0.17 to 0.19 with small sentence encoders, and 0.109 with the embedder used in LangPatch. JAMIE, which trains modality-specific encoders, obtains 0.208.

LangPatch generalizes zero-shot to another laboratory's feature set, which is infeasible for models with numerical encoders. The input layer of a numerical encoder is tied to the feature set of its training cohort, so applying it to another laboratory's features requires a manually constructed feature correspondence or retraining. LangPatch, trained on one laboratory's 39 features, aligns recordings described by another laboratory's 29 features without either: family label-transfer accuracy is 0.51 against a permutation null of 0.19 (W2 (b)). It also transfers zero-shot across cortical areas (about 70% of the in-domain correlation retained) and across species (47% from mouse to human and 63% from human to mouse).

The full method outperforms each of its components used alone, the criterion you propose. Each pair of rows uses an identical readout head (Metrics: Pearson r / raw MAE / AUROC on held-out cells, five-fold means). AUROC is computed per feature: held-out cells are split at the feature's median, and the prediction is scored on how well it separates the upper half from the lower half (0.5 is chance).

Visual cortexMotor cortex
GenePT alone (Plain GenePT MLP, Table 1)0.505 / 9.22 / 0.7680.502 / 8.02 / 0.790
LangPatch, same head0.522 / 9.05 / 0.7780.531 / 7.87 / 0.807
Contrastive learning alone (CLIP on numerical expression and numerical electrophysiology)0.419 / 10.61 / 0.7170.495 / 8.66 / 0.782
LangPatch, same head0.521 / 9.09 / 0.7770.532 / 7.90 / 0.807
GenePT + serialized records + InfoNCE, stacked without our components0.519 / 9.04 / 0.7800.506 / 7.92 / 0.791
LangPatch, same heads0.521 / 8.98 / 0.7790.521 / 7.65 / 0.802

LangPatch improves r over GenePT alone by 0.017 and 0.029, and over contrastive learning with numerical encoders by 0.10 and 0.04. Compared with a direct combination of the three existing components (third pair), the components we introduce reduce the alignment error in every fold of both cohorts, from 0.118 to 0.109 on visual cortex and from 0.170 to 0.139 on motor cortex.

Another benefit is that the framework is easy to extend to additional modalities: in principle, the same frozen text embedder and contrastive objective apply to any modality that can be verbalized as named quantities, such as morphological features.

W2 and Q2. The benefit of verbalization

We have run both requested experiments: a comparison with an MLP encoder on quintile-binned features (a), and a test with different feature vocabularies (b).

(a) MLP encoder on quintile-binned features (Q2). The baseline encodes the recording with an MLP on one-hot quintile bins and the transcriptome with plain GenePT embeddings (no adapter). The objective, training schedule, folds and readout head are identical to those of LangPatch.

Model (same readout head)Visual cortex r / MAE / AUROCMotor cortex r / MAE / AUROC
LangPatch0.521 / 9.09 / 0.7770.532 / 7.90 / 0.807
GenePT (no adapter) + MLP on quintile-binned features, same objective0.507 / 9.24 / 0.7670.495 / 8.17 / 0.783
the row above, trained with the contrastive loss alone (no auxiliary regression)0.516 / 9.13 / 0.7740.514 / 8.06 / 0.794

LangPatch outperforms this baseline on all three metrics in both cohorts.

(b) Cohorts with different feature vocabularies. The visual-cortex model was trained on recordings described by 39 Allen Institute (IPFX) features. We applied it to motor-cortex recordings described by the 29 features of Scala et al. (https://doi.org/10.1038/s41586-020-2907-3; no feature names in common), without retraining and without a feature correspondence. If the two vocabularies are mapped to a common space, a recording should lie close to the transcriptomes of its own cell family. We quantify this by label transfer between the recording and transcriptome embeddings of the same cells.

Zero-shot testCellsFamily label transferPermutation nullMajority classFOSCTTM (chance 0.50)
Visual-cortex model → motor-cortex recordings (29 features)1,2080.510.190.240.41
same, interneuron families (the classes the model was trained on)8170.780.300.350.39
Motor-cortex model → visual-cortex recordings (39 features)3,6540.650.310.420.41

All rows exceed the permutation null in all five folds (p = 0.001), which supports the claim. This setting is also infeasible for an MLP on binned features, whose input layer is tied to the feature set of its training cohort and therefore cannot be applied to the other laboratory's recordings.

(c) Choice of text embedder. We re-embedded the gene descriptions and recordings with ten alternative embedders and retrained LangPatch with all other settings fixed.

Text embedder (used for both modalities)FOSCTTM ↓ visualFOSCTTM ↓ motorForward MAE ↓ visualForward MAE ↓ motor
TF-IDF + SVD (bag of words, no language model)0.2250.2719.057.77
bge-base-en-v1.5 (open, 110M)0.1890.1939.047.70
all-MiniLM-L6-v2 (open, 22M)0.1750.2039.097.78
bge-large-en-v1.5 (open, 335M)0.1730.1699.027.73
text-embedding-ada-002 (API)0.1400.1599.067.69
bge-m3 (open, 568M)0.1390.1679.027.61
Qwen3-Embedding-8B (open)0.1240.1459.067.64
Cohere embed-5-pro (API)0.1210.1419.037.69
text-embedding-3-small (API)0.1160.1389.027.72
Qwen3-Embedding-4B (open)0.1150.1399.017.70
text-embedding-3-large (API; LangPatch)0.1090.1398.987.65

Alignment improves with the quality of the embedder (see W1), and the same trend holds in the cross-laboratory test of (b): the five embedders with the lowest FOSCTTM achieve a mean family label-transfer accuracy of 0.49 and 0.55 in the two directions, compared with 0.35 and 0.29 for the remaining six.

W3. Matched heads

With a matched readout head, the margin over Plain GenePT is unchanged on visual cortex and larger on motor cortex. In the table below, the LangPatch embedding is evaluated with each baseline's own head, using the same script and default hyper-parameters (a single shared multi-output MLP, without any selection).

HeadEmbeddingVisual cortex r / MAE / AUROCMotor cortex r / MAE / AUROC
Plain GenePT's head (2048–512)Plain GenePT (Table 1)0.505 / 9.22 / 0.7680.502 / 8.02 / 0.790
LangPatch0.522 / 9.05 / 0.7780.531 / 7.87 / 0.807
scFM baselines' head (512–256)Nicheformer (Table 1)0.486 / 9.56 / 0.7560.503 / 8.06 / 0.784
Geneformer (Table 1)0.492 / 9.54 / 0.7600.146 / 10.44 / 0.574
LangPatch0.521 / 9.09 / 0.7770.532 / 7.90 / 0.807

With the same head, LangPatch improves r over Plain GenePT by 0.017 on visual cortex and by 0.029 on motor cortex (0.017 and 0.018 in Table 1). The gain is therefore attributable to the representation rather than to the head.

Human cohort. With the mouse configuration, LangPatch achieves a higher AUROC (0.722 vs. 0.718 for JAMIE) and a lower MAE (10.00 vs. 10.03), with comparable r (0.400 vs. 0.406). On the two mouse cohorts, LangPatch outperforms JAMIE on 36 of 39 and 27 of 29 features.

Q1. Main methodological contribution, and ablation on each component of LangPatch

The main methodological contribution is the language interface. Replacing it with numerical encoders for both modalities reduces r from 0.521 to 0.419 on visual cortex and from 0.532 to 0.495 on motor cortex (W1, second pair).

Within this interface, the component we introduce is the context adapter. The table below ablates the adapter with all other settings fixed:

Visual cortex r / MAEMotor cortex r / MAEFOSCTTM ↓ visualFOSCTTM ↓ motorFamily label transfer ↑ visualmotor
LangPatch0.521 / 8.980.521 / 7.650.1090.1390.9270.822
without the adapter0.516 / 9.070.501 / 7.900.1100.1590.9210.745
adapter with label-free statistics only (Q3)0.519 / 9.070.505 / 7.920.1130.1510.9120.780
without all four added components0.519 / 9.040.506 / 7.920.1180.1700.9150.800

Removing the adapter degrades prediction on 31 of 39 visual-cortex features and 23 of 29 motor-cortex features (p < 0.001 for both). Its effect increases as the cohort size decreases: on motor cortex (1,208 cells), removing it reduces r from 0.521 to 0.501 and family label-transfer accuracy from 0.822 to 0.745, and on the human cohort (612 cells) the alignment error increases from 0.283 to 0.312.

Regarding the 0.133/0.186 control cited in S2: that control was trained with a reduced training recipe and therefore differs from the full method in more than the adapter. The row above ablates only the adapter under the full recipe, and the revised paper reports this row instead.

Q3. Adapter with label-free statistics only

Most of the gain is retained. With an adapter restricted to label-free statistics, FOSCTTM is 0.113 on visual cortex and 0.151 on motor cortex, compared with 0.109 and 0.139 for the full adapter and 0.208 and 0.222 for JAMIE.

Changes in the revised PDF. Sec. 5.4 with Table 2 and Fig. 4 (new): comparison of the full method with its components, matched-head comparison, adapter ablation with the label-free adapter, MLP baseline on binned features, text-embedder ablation and cross-laboratory test. Sec. 5.3: cross-laboratory test. Appendix A (new): all component removals and hyper-parameter sensitivity (Table 3), text-embedder table (Table 4). Sec. 1, Sec. 2 and the overview of Sec. 3: statement of novelty as in W1.

We hope that these results address your concerns regarding novelty, verbalization and matched experimental settings, and we would be grateful if you would reconsider your rating in light of them.

Comments on the paper

1 comment

We sincerely thank all reviewers and the Area Chair for their thoughtful feedback, and we deeply appreciate your recognition of our paradigm-level innovation (Reviewers wFFJ, EBFF, gm6o). At the same time, we fully agree that in terms of methodological rigor, thoroughly validating how our combined methods work together, and isolating why each part is necessary, was a key area where our initial manuscript needed improvement. Following your constructive suggestions, we have answered every reviewer question in detail and completed comprehensive ablations alongside all requested control and transfer experiments on the exact five-fold splits of the paper, in addition to rewriting Section 5 for clarity and releasing all underlying CSV result files and regeneration scripts. Together, we believe these new results substantially address the limitations in both our empirical validation and our technical novelty. Detailed tables and analyses are provided in our individual responses below and in the revised manuscript.

New results in summary

  1. Language space versus numerical encoders (wFFJ; EBFF Q2). Under an identical readout head, LangPatch outperforms contrastive learning with numerical encoders for both modalities (r = 0.521 vs. 0.419 on visual cortex and 0.532 vs. 0.495 on motor cortex), as well as every mixed text/numerical variant in the comparison.

  2. Matched readout heads (EBFF W3). Evaluated with Plain GenePT's own head, the LangPatch embedding achieves r = 0.522 on visual cortex and 0.531 on motor cortex, compared with 0.505 and 0.502 for Plain GenePT. The margin is unchanged from Table 1 on visual cortex and larger on motor cortex.

  3. Component ablation and hyper-parameter sensitivity (gm6o W2; EBFF Q1). The context adapter contributes most: removing it degrades prediction on 31 of 39 visual-cortex features and 23 of 29 motor-cortex features (p < 0.001 for both). The method is also robust to its hyper-parameters. When each of the four is varied individually, the alignment error (FOSCTTM; 0 is perfect, 0.5 is chance) remains between 0.102 and 0.117 on visual cortex and between 0.131 and 0.154 on motor cortex, compared with 0.208 and 0.222 for JAMIE.

  4. Text-embedder ablation (wFFJ; EBFF W2). Across eleven text embedders, FOSCTTM decreases from 0.225 on visual cortex and 0.271 on motor cortex with a bag-of-words representation to 0.109 and 0.139 with the embedder used in LangPatch.

  5. Zero-shot generalization to another laboratory's feature set (wFFJ; EBFF W2). A model trained on recordings described by 39 Allen Institute features aligns recordings described by a different laboratory's 29 features, without retraining and without a feature correspondence. Family label-transfer accuracy is 0.51 against a permutation null of 0.19, and 0.78 against 0.30 on interneuron families. Models with numerical encoders are not applicable to such inputs.

  6. New Human → mouse transfer experiment (wFFJ). A model trained on 612 human neurons predicts mouse electrophysiology zero-shot with a mean r of 0.39. Of 24 concepts, 23 exceed the permutation null, and 63% of the in-domain correlation is retained (47% in the mouse → human direction in the original paper, so human-to-mouse transfer is much better).

The novelty of LangPatch lies in how its components are combined: a single frozen text embedder shared by both modalities, verbalized recordings, a context adapter and contrastive alignment. The new experiments show that this combination is effective: it outperforms contrastive learning with numerical encoders (result 1), GenePT alone under a matched readout head (result 2) and the same model without the context adapter (result 3), and its alignment improves with the quality of the text embedder (result 4).

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.