acceptodds
Under review as a conference paper at ICLR 2027

How Should We Describe a Gene? Learning Biological Representations for Perturbation Prediction

Abstract

Predicting responses to unseen genetic perturbations requires representing perturbed genes using prior knowledge. Text provides a natural interface to this knowledge, but existing representations rely on fixed, hand-designed descriptions, leaving unclear what to include or how to organise it. We learn a shared description protocol for unseen genes, separating the benefits of richer knowledge from those of its representation. An LLM iteratively revises the protocol using held-out prediction and trial history while the prediction pipeline remains fixed. We predict transcriptome-wide log-fold changes and introduce *Continuous Directed Overlap*, a rank- and direction-aware metric focused on recovering the strongest effects. Across four Perturb-seq datasets, richer knowledge consistently improves prediction, structured descriptions add further gains, and search generally improves validation performance. In our evaluations, the resulting representations outperform the tested gene embeddings across multiple predictors, retain utility across cellular contexts, and enhance predictions from established single-cell generative models.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
gene representations, genetic perturbation prediction, LLM-guided search

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_rn5yOpenReview ↗
Rating
4(+1)
Confidence
3
Informativeness
3(+1)
Presentation
2
Soundness
3
Read review
Paper summary

This paper examines how to organize existing biological knowledge into gene representations for predicting responses to unseen single-gene perturbations. The authors construct gene descriptions from public resources and encode them as text embeddings. They introduce an LLM-guided search procedure that iteratively refines a shared gene-description protocol based on validation performance in perturbation prediction.

The largest performance gains come from richer biological knowledge. More structured descriptions provide further improvements, while LLM-guided search adds a smaller but consistent benefit. The resulting text representations transfer across different predictors and models. The paper also uses transcriptome-wide log-fold changes from DESeq2 as prediction targets and introduces Continuous Directed Overlap (CDO), a metric that emphasizes the ranking of strongly responding genes and agreement in response direction.

Critical strengths
  1. The paper uses a clear, controlled design to assess the effects of gene knowledge and description quality. It progressively compares gene names, NCBI/HGNC metadata, WikiCrow-enriched information, and canonical descriptions while keeping the encoder and predictor fixed.

  2. The canonical rich-text representation consistently outperforms existing gene embeddings across datasets and metrics, even without protocol search. This supports its practical utility and suggests that the gains are not confined to the proposed CDO metric or a single search run.

Critical weaknesses

1. The representation-learning motivation is insufficiently connected to existing causal representation learning literature

The paper asks which gene representations best support prediction of unseen perturbations and emphasizes the generalization limits of foundation models and fixed gene embeddings. However, the related-work discussion focuses mainly on general-purpose or pretrained representations. It gives limited attention to causal representation learning methods developed for perturbation prediction, such as Discrepancy-VAE [1], sVAE+ [2], SAMS-VAE [3], and PerturbedVAE [4] . PerturbedVAE is particularly relevant in both motivation and problem setting, as it also examines whether foundation models learn perturbation-relevant representations that generalize to unseen perturbations.

The results demonstrate the usefulness of richer biological text embeddings, but the paper does not clearly explain how its approach differs from these causal representation learning methods in objective, assumptions, and generalization mechanism.

2. The empirical validation of CDO as a new evaluation metric remains limited

The paper presents CDO as a major methodological contribution for evaluating the ranking and directional consistency of strongly responding genes. The authors provide synthetic corruption tests, sensitivity analyses, and results across four Perturb-seq datasets, but validation remains largely confined to the paper’s own task and model settings.

Although the paper discusses limitations of Pearson correlation, MAE, top-K overlap, DEG recovery, and perturbation discrimination, it does not systematically compare CDO with existing perturbation evaluation metrics. It also remains unclear when these metrics disagree on model rankings, which failure modes each captures, and which better reflects biological relevance.

The results suggest that CDO has reasonable design properties, but do not yet demonstrate that it is more discriminative or reliable than existing metrics in practical perturbation prediction.

[1] Zhang J, Greenewald K, Squires C, et al. Identifiability guarantees for causal disentanglement from soft interventions[J]. Advances in Neural Information Processing Systems, 2023, 36: 50254-50292.

[2] Lopez R, Tagasovska N, Ra S, et al. Learning causal representations of single cells via sparse mechanism shift modeling[C]//Conference on Causal Learning and Reasoning. PMLR, 2023: 662-691.

[3] Bereket M, Karaletsos T. Modelling cellular perturbations with the sparse additive mechanism shift variational autoencoder[J]. Advances in Neural Information Processing Systems, 2023, 36: 1-12.

[4] Jiang W, Liu Y, Cai Y, et al. What Makes a Representation Good for Single-Cell Perturbation Prediction?[J]. arXiv preprint arXiv:2605.19343, 2026.

Questions for authors

Q1: Could the authors expand the discussion of causal or intervention-aware representation learning, including Discrepancy-VAE, sVAE+, SAMS-VAE, and PerturbedVAE? How does the proposed text-based approach differ from methods that explicitly learn perturbation-relevant or mechanism-aware representations in problem setting, learning objective, and generalization to unseen or OOD perturbations?

Q2: Could the authors systematically compare CDO with existing metrics, including DEG recovery, Jaccard/F1, and perturbation discrimination, and assess whether it adds useful information across more models, datasets, or real failure cases? When CDO ranks models differently from Pearson, MAE, or DEG-based metrics, can the authors show that its rankings better reflect biologically meaningful recovery of strong responses?

6 comments
AuthorsOpenReview ↗

Response to Reviewer rn5y (Part 1)

We thank the reviewer for their careful assessment and constructive suggestions. We particularly appreciate their recognition of our controlled representation comparisons and the utility of the resulting embeddings. We summarise our responses below:

  • W1/Q1 – Causal representation learning: We agree that Discrepancy-VAE, sVAE+, SAMS-VAE and PerturbedVAE are important related methods. While these approaches learn representations of cellular mechanisms and intervention effects, they do not, in their current formulations, support zero-shot prediction for genes never perturbed during training. Their generalisation capabilities instead include previously unseen combinations of known interventions. Our approach directly addresses unseen gene perturbations by constructing representations from external biological knowledge, without requiring experimental responses for those genes. We will expand the Related Work accordingly.

  • W2/Q2 – Validation of CDO: We evaluated DEG recovery, signed strong-response overlap, prediction error and perturbation discrimination on held-out K562 perturbations. On 218 test perturbations, CDO increases from 0.1546 to 0.1777, while MAE@100 decreases from 0.5095 to 0.4423 and DEG AP increases from 0.1397 to 0.1645, even though Corr@100 does not improve.

  • W2/Q2 – Broader metric comparisons: Across ten representations evaluated on the 184-perturbation embedding-covered subset of the same K562 holdout, optimised text achieves the best point estimates on all seven metrics. Improvements in signed Jaccard also persist at 1.25-, 1.5- and 2-fold response thresholds.

We provide further details and results below.

W1/Q1: Relationship to causal representation learning

We thank the reviewer for highlighting Discrepancy-VAE, sVAE+, SAMS-VAE and PerturbedVAE. We agree that these are important contributions to perturbation modelling and will expand our Related Work section accordingly.

These approaches investigate how cellular states and intervention effects can be represented or inferred from experimental observations. Discrepancy-VAE studies causal disentanglement and identifiability under soft interventions; sVAE+ models sparse mechanism shifts; SAMS-VAE models sparse, composable perturbation effects; and PerturbedVAE separates perturbation-invariant and responsive latent factors while incorporating causal structure. These methods offer important advantages in learning structured representations of cellular mechanisms and modelling intervention effects. Several, including SAMS-VAE and PerturbedVAE, also generalise to unseen combinations of known interventions.

However, these methods do not, in their current formulations, support zero-shot prediction of the response to perturbing an entirely unseen gene. SAMS-VAE and PerturbedVAE can generalise to unseen combinations of interventions, but only when their constituent effects have been learned during training. The authors of PerturbedVAE explicitly acknowledge this limitation, identifying additional gene-level information, including biological annotations or gene embeddings, as a potential route to unseen-gene prediction (Appendix B). sVAE+ also evaluates unseen interventions, but through transfer learning, fine-tuning intervention-specific parameters using expression data from the held-out perturbations. Thus, while these approaches offer valuable forms of generalisation, they cannot directly predict the effects of previously unperturbed genes from their biological identity alone.

Our work addresses this challenge by constructing representations directly from prior biological knowledge about the intervened gene. We optimise a shared, human-readable gene-description protocol using downstream prediction feedback, while keeping the description generator, encoder and predictor fixed. This enables predictions for genes with no experimental perturbation responses in training. Unlike the causal approaches above, however, we do not explicitly learn latent cellular mechanisms or claim causal identifiability.

We therefore see considerable potential in combining the strengths of both approaches. Our matched benchmarks, consistent with Cole et al. (2026), demonstrate the predictive value of knowledge-derived gene representations, although they do not directly compare against these causal models. Incorporating such representations, potentially enriched with interaction networks and pathway information, as priors within causal or intervention-aware models could provide a means of generalising to entirely unseen gene targets while retaining their ability to learn structured cellular mechanisms and compositional perturbation effects. We will expand the Related Work and Discussion to clarify this distinction and the opportunities for integration.

AuthorsOpenReview ↗

Response to Reviewer rn5y (Part 2)

W2/Q2: Additional evaluation of CDO

We agree that CDO should be compared systematically with existing metrics. Our new evaluations show that improvements selected using CDO are accompanied by better DEG recovery, strong-response overlap, prediction accuracy and perturbation discrimination, even when correlation remains unchanged.

In the 218-perturbation K562 holdout (see response to FXRR), history-guided search improves CDO over canonical from 0.1546 to 0.1777 (paired Δ +0.0231 [0.0162, 0.0305]). MAE@100 decreases from 0.5095 to 0.4423, DEG AP increases from 0.1397 to 0.1645, and signed Jaccard from 0.1056 to 0.1376. PDS-L1 increases from 0.6291 to 0.7177 and PDS-cosine from 0.6445 to 0.7290. Corr@100 remains essentially unchanged.

We also evaluated GenePT, UniProt, ESM2, Geneformer, scGPT, GeneCompass and scFoundation on the 184-perturbation embedding-covered subset of the original 218 test perturbations, with 744 corresponding development perturbations. We reused protocols selected using the full 874-perturbation development set, without further optimisation, and refitted Ridge on the restricted development set. No test responses informed these new protocol searches or Ridge fits.

Prediction performance

Gene representationCDO ↑Corr@100 ↑MAE@100 ↓
Mean response0.1105 [0.1041,0.1169]0.5955 [0.5625,0.6265]0.6152 [0.5747,0.6597]
GenePT0.1371 [0.1270,0.1473]0.6841 [0.6525,0.7140]0.5540 [0.5160,0.5946]
UniProt0.1266 [0.1184,0.1351]0.6734 [0.6461,0.6993]0.5720 [0.5320,0.6150]
ESM20.1206 [0.1132,0.1283]0.6564 [0.6276,0.6835]0.5852 [0.5462,0.6274]
Geneformer0.1183 [0.1106,0.1261]0.5917 [0.5566,0.6248]0.6045 [0.5642,0.6481]
GeneCompass0.1240 [0.1157,0.1325]0.6297 [0.5974,0.6600]0.5896 [0.5494,0.6339]
scGPT0.1292 [0.1200,0.1389]0.6364 [0.6052,0.6659]0.5809 [0.5398,0.6261]
scFoundation0.1104 [0.1041,0.1168]0.5965 [0.5639,0.6274]0.6150 [0.5744,0.6594]
Ind.-selected text0.1452 [0.1351,0.1556]0.7186 [0.6916,0.7441]0.5324 [0.4943,0.5734]
K562-selected text0.1703 [0.1558,0.1851]0.7232 [0.6915,0.7528]0.4618 [0.4270,0.4986]

DEG recovery, strong-response overlap and perturbation discrimination

Gene representationDEG AP ↑Signed Jaccard ↑PDS-L1 ↑PDS-cosine ↑
Mean response0.1014 [0.0848,0.1195]0.0743 [0.0700,0.0788]0.5000 [0.4580,0.5419]0.5000 [0.4584,0.5419]
GenePT0.1216 [0.1013,0.1441]0.0929 [0.0861,0.0995]0.5804 [0.5372,0.6221]0.5933 [0.5489,0.6363]
UniProt0.1160 [0.0965,0.1374]0.0869 [0.0806,0.0935]0.5617 [0.5182,0.6036]0.5744 [0.5317,0.6167]
ESM20.1108 [0.0922,0.1314]0.0813 [0.0760,0.0865]0.5415 [0.4985,0.5832]0.5515 [0.5089,0.5938]
Geneformer0.1054 [0.0878,0.1246]0.0855 [0.0794,0.0919]0.5140 [0.4724,0.5564]0.5217 [0.4789,0.5644]
GeneCompass0.1117 [0.0925,0.1328]0.0854 [0.0798,0.0912]0.5353 [0.4926,0.5781]0.5408 [0.4979,0.5840]
scGPT0.1155 [0.0957,0.1374]0.0901 [0.0833,0.0972]0.5447 [0.5018,0.5880]0.5581 [0.5143,0.6017]
scFoundation0.1015 [0.0849,0.1196]0.0743 [0.0700,0.0787]0.5002 [0.4583,0.5421]0.5004 [0.4588,0.5422]
Ind.-selected text0.1299 [0.1088,0.1531]0.0983 [0.0911,0.1054]0.6143 [0.5706,0.6568]0.6325 [0.5889,0.6747]
K562-selected text0.1530 [0.1295,0.1780]0.1308 [0.1174,0.1446]0.7144 [0.6736,0.7543]0.7293 [0.6859,0.7696]

Values are means [95% perturbation-bootstrap confidence intervals] on the 184-perturbation test panel. DEG AP evaluates recovery of statistically significant DEGs; signed Jaccard evaluates overlap of responses exceeding a 1.5-fold threshold with matching direction; PDS-L1 and PDS-cosine measure perturbation discrimination using predicted and observed LFC vectors.

Optimised text achieves the best point estimates across all seven metrics. Relative to GenePT, the strongest existing embedding by CDO, DEG AP improves from 0.1216 to 0.1530, signed Jaccard from 0.0929 to 0.1308, and PDS-L1 from 0.5804 to 0.7144, with paired confidence intervals excluding zero.

Comparing independently selected and history-guided text, CDO increases from 0.1452 to 0.1703 (paired difference +0.0251 [0.0173, 0.0332]) and MAE@100 decreases from 0.5324 to 0.4618 (−0.0706 [−0.0851, −0.0570]). Corr@100 changes only from 0.7186 to 0.7232 (+0.0047 [−0.0083, 0.0176]), without a statistically resolved improvement.

Why does Corr@100 improve less? Corr@100 measures linear agreement within the 100 strongest observed responses, without assessing whether those genes rank highly among all predicted responses. It is also insensitive to positive rescaling and constant offsets. CDO instead evaluates recovery of strong responses in the correct direction and relative order, penalising displacement by other predicted responses. Improved ranking and effect-size accuracy can therefore coexist with unchanged correlation. The accompanying improvements in all metrics suggested by the reviewer show that the gains are not confined to CDO.

AuthorsOpenReview ↗

Response to Reviewer rn5y (Part 3)

Sensitivity to response thresholds. Because binary recovery metrics depend on how a strong response is defined, we repeated the signed Jaccard analysis at 1.25-, 1.5- and 2-fold expression-change thresholds, without changing the selected protocols or predictors. On the same 184-perturbation test panel:

Fold-change thresholdCanonicalLLM-guided searchPaired improvement [95% CI]
1.25-fold0.23330.2470+0.0137 [0.0043,0.0234]
1.5-fold0.09890.1308+0.0319 [0.0227,0.0417]
2-fold0.05950.0980+0.0385 [0.0243,0.0526]

The improvement remains statistically resolved at every threshold and increases for stricter response definitions, supporting improved recovery of substantial transcriptional changes. This analysis assesses sensitivity of the binary overlap metric, rather than changing CDO's parameters.

Our claim is not that CDO replaces existing metrics or universally yields better model rankings. Rather, it provides a complementary rank- and direction-aware measure focused on strong transcriptional responses. The additional experiments show that improvements identified using CDO are accompanied by better prediction error, DEG recovery, strong-response overlap and perturbation discrimination.

We will incorporate the additional CDO evaluations and expanded discussion of causal representation learning into the revised manuscript. We thank the reviewer again for their constructive suggestions, which have helped us strengthen the empirical validation of CDO and clarify how our approach complements existing causal representation-learning methods.

Reviewer_FmG4OpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The authors suggest a language based representation of genes for perturbation modeling. They convert an LLM language descriptor to an embedding vector (using pretrained text encoder). They use a frontier model as a "search LLM” which defines the strategy/protocol (human readable) which guides an open source LLM to construct the embedding using the guidance and available resources. They ultimately show that the amount and organization of the knowledge impacts the performance of perturbation prediction.

Critical strengths

The paper is clearly written and understandable, the authors provide thorough evaluations and performance analysis.

Critical weaknesses

1/ Novelty of findings and Contribution; there have been several recent works highlighting the importance of gene representation / embedding for the perturbation prediction task, e.g., TxPert (Wenkel et al., 2026), and PRESAGE (Littman et al., 2025), therefore the main contribution / finding is unclear (putting aside the different metrics suggested).

2/ The LLM based workflow; The LLM guided optimization suggests this is a sub-optimal approach. Looking at the steps presented in Figure 2; some of the steps are naive and would have been provided initially by any scientist (e.g. adding cell type) therefore it seems like simply starting from a human based protocol could outperform this choice. Furthermore, as depicted in Table 1, simply combining all the representations reaches almost identical results.

Questions for authors

In light of the weaknesses above can the authors

  1. Provide better motivation for the necessity and value of the LLM based optimization (given the minimal gain)
  2. More broadly, it will be valuable to clarify what they see as the contribution of this work– both:
    a. the importance of the work to the single-cell perturbation modeling community?
    b. the relevance for the broader ML community and fit to ICLR.
4 comments
AuthorsOpenReview ↗

Response to Reviewer FmG4 (Part 1)

We thank the reviewer for their assessment and for recognising the thoroughness of our empirical evaluation. We appreciate the concerns regarding novelty and the value of LLM-guided optimisation. We summarise our response below:

  • W1/Q2 – Novelty and contributions: While TxPert and PRESAGE demonstrate the importance of biological knowledge, they largely take source representations as given. We instead optimise how that knowledge is selected, compressed and organised before embedding, using a shared, human-readable description protocol. Our representations also improve established prediction architectures without modifying their training procedures.

  • W2/Q1 – Value of LLM-guided optimisation: Our canonical protocol was already expert-designed. In a new matched-budget comparison of 40 proposals per strategy, history-guided search achieves CDO 0.1777, compared with 0.1546 for canonical and 0.1540 for independent proposals, on 218 held-out K562 perturbations. This is a 15.0% improvement over canonical, with a paired 95% confidence interval excluding zero.

  • Q2 – Broader ML relevance: We investigate whether the information presented to a fixed encoder can itself be optimised using downstream task feedback, providing an interpretable approach to representation construction with applications beyond perturbation prediction.

We provide the detailed discussion and results below.

W1/Q2: Novelty, contributions and relationship to prior work

We agree that TxPert (Wenkel et al., 2026) and PRESAGE (Littman et al., 2025) demonstrate the importance of biological prior knowledge for perturbation prediction. TxPert investigates how biological interaction graphs can be combined within predictive models, while PRESAGE learns to integrate embeddings derived from diverse knowledge sources. We view these works as complementary to ours.

Our distinction is that these approaches largely treat source representations as fixed inputs, whereas we optimise how the underlying knowledge is expressed before embedding. Although PRESAGE learns how to combine pretrained embeddings, it does not explicitly optimise which biological details within those sources should be retained, compressed, emphasised or omitted. This distinction is particularly relevant as biological knowledge becomes increasingly extensive and overlapping. Indeed, our information hierarchy demonstrates that adding biological knowledge improves prediction, but also that organising the same available information into structured descriptions can provide further gains. Our approach therefore complements knowledge-source selection by optimising how the information within those sources is represented.

We use an LLM to select, synthesise and organise supplied biological metadata through a shared, human-readable gene-description protocol, optimised using downstream prediction feedback. The knowledge sources, description generator, encoder and predictor remain fixed during search, allowing us to isolate the contribution of representation construction independently of additional knowledge or changes to model architecture.

Our contributions are therefore: (i) a controlled decomposition of biological knowledge, its organisation and subsequent optimisation; (ii) LLM-guided optimisation of a reusable description protocol applicable to unseen genes; (iii) competitive, transferable gene representations that improve established prediction models; and (iv) transcriptome-wide LFC prediction with CDO as a complementary evaluation metric.

Importantly, our representations provide a modular input to a broad class of perturbation-prediction models that use pretrained gene embeddings. Their utility is therefore not restricted to our downstream predictor. We demonstrate this directly by replacing the native GenePT embeddings in scLAMBDA and ESM2 embeddings in CellFlow, improving prediction performance across metrics without modifying either model's architecture or training procedure. This illustrates how improvements in representation construction can benefit existing models independently of their predictive architectures.

Our approach is also compatible with the broader knowledge-integration strategies explored by TxPert and PRESAGE. Interaction graphs, screening data and other biological sources could be incorporated into richer gene descriptions, while the resulting optimised embeddings could serve as inputs to models that consume gene-level representations. We therefore view these approaches as complementary rather than competing.

AuthorsOpenReview ↗

Response to Reviewer FmG4 (Part 2)

W2/Q1: Value of LLM-guided optimisation

Our canonical protocol was already expert-designed, prioritising molecular function, interaction partners, cellular context and consequences of gene disruption within 60–100 words. It was not deliberately chosen as a weak starting point.

We appreciate that some accepted revisions in Figure 2 may appear straightforward in retrospect. However, biological plausibility does not guarantee predictive utility, and it was not obvious in advance which changes would be beneficial. For example, in the RPE1 search (Figure 2), the first accepted revision relaxed the canonical 60–100-word limit, while the largest individual improvement (+1.11% CDO) came from incorporating mechanism-linked disease associations, which our original mechanism-focused protocol had omitted. Yet most other plausible proposals were rejected. This illustrates the value of empirically evaluating which biological details should be retained, emphasised or omitted, rather than assuming their usefulness based on expert intuition alone.

Figure 2 provides an interpretable record of this process, showing how successive proposals are evaluated and accepted or rejected based on downstream performance. Nevertheless, a single trajectory does not establish whether using the results of previous evaluations is more effective than independently generating plausible alternatives. To test this directly, we conducted an additional matched-budget experiment comparing independent proposal generation with history-guided optimisation. Starting from the same canonical protocol, both strategies evaluated 40 candidate protocols, with selection based on cross-validation over 874 (80%) development perturbations and final evaluation on 218 (20%) held-out K562 perturbations:

ProtocolTest CDO ↑Test MAE@100 ↓
Canonical0.15460.5095
Independent0.15400.5114
History-guided search0.17770.4423

History-guided search improves CDO by 15.0% relative to canonical (paired ΔCDO +0.0231, 95% CI [0.0162, 0.0305]) and reduces MAE@100 by 13.2%. By contrast, independently generated candidates provide no improvement over canonical despite the same evaluation budget. The improvements also extend to DEG recovery, strong-response overlap and perturbation discrimination, as detailed in our responses to Reviewers FXRR and rn5y.

These results provide evidence that using empirical feedback to refine a shared description protocol offers value beyond independently generating plausible alternatives. We acknowledge that this comparison currently covers one dataset and one search LLM, and does not establish superiority over every possible expert-designed protocol. We intend to extend this evaluation to additional datasets and search models.

Regarding Table 1, we agree that combining the available biological metadata already produces a strong representation. However, organising the same information using our canonical protocol improves CDO on RPE1, HepG2 and Jurkat, with paired confidence intervals excluding zero. The new held-out experiment further demonstrates that optimising the protocol can provide additional gains beyond an expert-designed starting point, without adding new knowledge sources.

Q2: Broader ML relevance and fit to ICLR

We agree that the broader representation-learning contribution should be made more explicit. Many existing approaches learn how to encode, combine or predict from available representations. Our work investigates a complementary question: can the information presented to a fixed encoder itself be optimised using downstream task feedback such that the learned representation remains a human-readable protocol that can be inspected, edited and reused?

This is particularly relevant to scientific domains where extensive prior knowledge exists, but that knowledge is heterogeneous, redundant or difficult to represent effectively. A shared natural-language protocol provides an interpretable mechanism for selecting, synthesising and compressing this information, while enabling systematic evaluation of which representation choices improve predictive performance.

We believe this connects directly to representation learning, LLM-guided optimisation and data-efficient scientific machine learning, with clear application beyond biology, including materials discovery and scientific experimental design.

We thank the reviewer again for their comments and will revise the Introduction and Discussion to clarify these contributions and their relationship to prior work.

Reviewer_FmG4OpenReview ↗

Rebuttal response

Thank you for taking the time and providing a detailed response; to better understand the new experiments and motivation / contribution presentation -- can the authors share the revised text?

During the rebuttal period you are allowed to upload a revised PDF, with an additional page for the main submission. See details here.

This will help judging the response.

Thank you in advance!

AuthorsOpenReview ↗

Reply to Reviewer FmG4

Thank you again for your feedback and for suggesting that we share the revised manuscript. We have now uploaded an updated PDF with the changes highlighted, including the additional experiments and clarifications discussed in our response. We hope these revisions help address your concerns.

Reviewer_FXRROpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper investigates how biological knowledge about genes should be represented for predicting responses to unseen genetic perturbations. The authors propose an LLM-guided framework that iteratively optimizes a shared gene-description protocol based on held-out perturbation prediction performance. The generated descriptions are converted into gene embeddings and evaluated using several downstream predictors. The paper also introduces Continuous Directed Overlap (CDO), a rank- and direction-aware metric for evaluating perturbation-induced expression changes. Experiments on four Perturb-seq datasets show that richer biological information improves prediction, while structured descriptions and protocol search provide additional gains. The resulting representations also outperform several existing gene embeddings and improve predictions from two established single-cell models.

Critical strengths

S1. The paper investigates an important question in perturbation prediction, i.e., how biological knowledge should be selected and organized to construct useful representations for unseen genes. Optimizing a shared gene-description protocol rather than individual descriptions is an interesting approach, and separating the amount of biological information from how it is represented provides a useful perspective on gene representation learning.

S2. The experiments are well organized, with comparisons across different knowledge sources, gene representations, downstream predictors, and cellular contexts. The results show that richer biological information consistently improves prediction, and the proposed embeddings outperform several existing gene representations under matched evaluation settings. The experiments integrating the embeddings into scLAMBDA and CellFlow also suggest that these representations can benefit different perturbation prediction architectures.

Critical weaknesses

W1. My main concern is the lack of an independent evaluation of the LLM-guided protocol optimization. The same 9-fold perturbation splits are repeatedly used to evaluate and select candidate protocols during the search. Although the downstream predictions are out-of-fold, the held-out responses still influence the selection of the description protocol. Therefore, improvements on these folds represent validation gains rather than independent evidence of generalization. The cross-dataset transfer experiments partially address this concern, but the improvements are relatively small, and none of the six cross-context CDO comparisons in Table 1C has a paired 95% confidence interval excluding zero. An independent perturbation-level test set that is never used during protocol optimization would provide stronger evidence for the generalization of the learned descriptions.

W2. The contribution of the proposed LLM-guided search is not sufficiently isolated from the benefits of using richer biological knowledge and structured descriptions. In Table 1, most of the improvement comes from adding biological metadata and applying the canonical description protocol, while the additional gains from protocol optimization are relatively modest. The paper compares different search LLMs but does not include simpler search strategies under the same evaluation budget, such as independently sampled candidate protocols or random modifications of the canonical protocol. Without these comparisons, it is difficult to determine whether the history-guided iterative search is necessary or whether similar improvements could be obtained through simpler candidate selection.

W3. The comparison with existing gene representations could be more comprehensive. Although the paper includes GenePT, ESM2, scGPT, and Geneformer, several relevant pretrained gene embedding methods are missing, particularly scFoundation and GeneCompass. GeneCompass is especially relevant because it incorporates biological prior knowledge into gene representation learning, which is closely related to the motivation of this work. Comparing these embeddings under the same downstream predictors and perturbation splits would provide a more complete assessment of the advantages of the proposed text-based representations.

Questions for authors

Q1. Can the authors evaluate the final optimized protocols on an independent set of held-out perturbations that is never used during protocol search? This would help establish whether the improvements generalize beyond the validation folds used for protocol selection.

Q2. How does the proposed history-guided search compare with simpler protocol selection strategies under the same budget of 40 candidate evaluations? For example, would independently generated candidate protocols or random modifications of the canonical protocol achieve similar improvements?

Q3. Can the authors extend the gene representation comparison to additional pretrained gene embeddings, particularly scFoundation and GeneCompass, using the same downstream predictors and perturbation splits? These comparisons would help assess whether the proposed representations provide advantages over other gene embeddings learned from large-scale expression data or biological prior knowledge.

3 comments
AuthorsOpenReview ↗

Response to Reviewer FXRR (Part 1)

We thank the reviewer for their constructive assessment and for recognising the value of studying how biological knowledge is selected and organised for perturbation prediction. We appreciate the specific suggestions for strengthening the evaluation and have conducted several additional experiments addressing all three weaknesses. To summarise:

  • W1/Q1 – Independent evaluation: We conducted new protocol-optimisation campaigns using 874 development perturbations and evaluated the selected protocols on 218 held-out K562 perturbations. History-guided search improves CDO from 0.1546 to 0.1777, a 15.0% relative improvement (paired ΔCDO +0.0231, 95% CI [0.0162, 0.0305]).

  • W2/Q2 – Contribution of LLM-guided search: Under the same budget of 40 evaluations, independently generated protocol candidates achieve CDO 0.1540, compared with 0.1777 for history-guided search. Improvements also extend to MAE, DEG recovery, strong-response overlap and perturbation discrimination.

  • W3/Q3 – Additional embeddings: We have added GeneCompass and scFoundation to a matched benchmark on 184 held-out K562 perturbations. Our optimised text achieves CDO 0.1703, compared with 0.1240 and 0.1104, respectively, with paired 95% confidence intervals excluding zero.

We provide the experimental details and full results below.

W1/W2 and Q1/Q2: Independent evaluation and contribution of LLM-guided search

We agree that an independent evaluation is important for distinguishing genuine generalisation from improvements on folds repeatedly used during protocol selection. We partitioned the K562 dataset at the perturbation level into 874 development perturbations (80%) and 218 test perturbations (20%). Test perturbations were excluded from predictor fitting and protocol selection during the new optimisation campaigns. Within the development set, candidate protocols were evaluated using 9 fold cross-validation, and the selected protocols were subsequently assessed on the held-out test set using Ridge regression.

To isolate the contribution of history-guided optimisation, we compared the best protocols identified within 40 candidate evaluations each of two strategies: independent proposal generation, in which the LLM receives no previous evaluation outcomes, and history-guided search, in which proposals are informed by the results of earlier trials. Both use the same canonical starting point, description-generation pipeline and downstream predictor.

MetricGene nameNCBI/HGNCCanonicalIndependentHistory-guided
CDO ↑0.1366 [0.1268, 0.1468]0.1475 [0.1374, 0.1577]0.1546 [0.1450, 0.1645]0.1540 [0.1444, 0.1639]0.1777 [0.1645, 0.1916]
Corr@100 ↑0.6713 [0.6430, 0.6990]0.7060 [0.6793, 0.7317]0.7311 [0.7062, 0.7553]0.7268 [0.7016, 0.7513]0.7286 [0.6990, 0.7570]
MAE@100 ↓0.5460 [0.5132, 0.5806]0.5282 [0.4954, 0.5633]0.5095 [0.4764, 0.5448]0.5114 [0.4782, 0.5466]0.4423 [0.4119, 0.4745]
DEG AP ↑0.1271 [0.1070, 0.1483]0.1331 [0.1126, 0.1547]0.1397 [0.1188, 0.1616]0.1393 [0.1184, 0.1612]0.1645 [0.1414, 0.1886]
Signed Jaccard ↑0.0946 [0.0873, 0.1024]0.1011 [0.0937, 0.1086]0.1056 [0.0987, 0.1130]0.1046 [0.0976, 0.1118]0.1376 [0.1252, 0.1504]
PDS-L1 ↑0.5754 [0.5357, 0.6142]0.6014 [0.5617, 0.6409]0.6291 [0.5895, 0.6679]0.6236 [0.5839, 0.6626]0.7177 [0.6806, 0.7530]
PDS-cosine ↑0.5791 [0.5376, 0.6187]0.6122 [0.5714, 0.6519]0.6445 [0.6054, 0.6835]0.6442 [0.6049, 0.6832]0.7290 [0.6892, 0.7672]

Values are means [95% perturbation-bootstrap confidence intervals] on 218 held-out K562 perturbations. DEG AP assesses recovery of significant differentially expressed genes; signed Jaccard measures directionally consistent strong-response overlap; PDS-L1 and PDS-cosine evaluate perturbation discrimination from LFC vectors.

History-guided search improves held-out CDO by 15.0% relative to canonical (paired ΔCDO +0.0231 [0.0162, 0.0305]) and outperforms independent proposal generation directly (+0.0237 [0.0168, 0.0311]). Its improvements extend to prediction error, DEG recovery, signed Jaccard and both perturbation-discrimination measures, although Corr@100 remains essentially unchanged. These complementary evaluations also address Reviewer rn5y's questions regarding CDO.

Importantly, independent proposal generation performs similarly to the canonical baseline (0.1540 versus 0.1546). Thus, the improvement cannot be attributed simply to evaluating additional independently proposed protocols. The results provide evidence that incorporating experimental history enables more effective refinement of the shared description protocol under the tested conditions.

Due to rebuttal time and computational constraints, the new evaluation covers K562 and one search LLM. We intend to extend the development/test evaluation to other datasets and search models in the revised work.

AuthorsOpenReview ↗

Response to Reviewer FXRR (Part 2)

W3 and Q3: Additional pretrained gene representations

We have added GeneCompass and scFoundation to our benchmark, using fixed gene-level embeddings from Tabares et al. (2026). To ensure matched gene coverage, we restricted the original 218-perturbation test set to 184 perturbations covered by the available embeddings, with 744 corresponding development perturbations. We reused the description protocols previously selected using the full 874-perturbation development set, without further optimisation, and refitted the downstream Ridge predictors on the coverage-restricted development set. None of the 184 test perturbations were used during protocol selection or predictor fitting. All representations use identical response targets and evaluation procedures.

Gene representationCDO ↑Corr@100 ↑MAE@100 ↓
Mean response0.1105 [0.1041, 0.1169]0.5955 [0.5625, 0.6265]0.6152 [0.5747, 0.6597]
GenePT0.1371 [0.1270, 0.1473]0.6841 [0.6525, 0.7140]0.5540 [0.5160, 0.5946]
UniProt0.1266 [0.1184, 0.1351]0.6734 [0.6461, 0.6993]0.5720 [0.5320, 0.6150]
ESM20.1206 [0.1132, 0.1283]0.6564 [0.6276, 0.6835]0.5852 [0.5462, 0.6274]
Geneformer0.1183 [0.1106, 0.1261]0.5917 [0.5566, 0.6248]0.6045 [0.5642, 0.6481]
GeneCompass0.1240 [0.1157, 0.1325]0.6297 [0.5974, 0.6600]0.5896 [0.5494, 0.6339]
scGPT0.1292 [0.1200, 0.1389]0.6364 [0.6052, 0.6659]0.5809 [0.5398, 0.6261]
scFoundation0.1104 [0.1041, 0.1168]0.5965 [0.5639, 0.6274]0.6150 [0.5744, 0.6594]
Ind.-selected text0.1452 [0.1351, 0.1556]0.7186 [0.6916, 0.7441]0.5324 [0.4943, 0.5734]
K562-selected text0.1703 [0.1558, 0.1851]0.7232 [0.6915, 0.7528]0.4618 [0.4270, 0.4986]

Values are means [95% bootstrap confidence intervals] on a common panel of 184 held-out K562 perturbations.

The K562-selected text representation achieves the best point estimate across all three metrics. Its paired CDO improvements over GeneCompass (+0.0462 [0.0332, 0.0594]) and scFoundation (+0.0599 [0.0457, 0.0743]) are statistically resolved. Its advantages also extend to DEG recovery and perturbation discrimination, as reported in our response to Reviewer rn5y.

Altogether, these results extend the findings of the original manuscript and demonstrate the competitiveness of optimised text representations relative to embeddings learned from large-scale expression data and biological prior knowledge.

We will incorporate the new holdout experiments and additional representation baselines into the revised manuscript. We thank the reviewer again for their suggestions, which have allowed us to strengthen the empirical evaluation and clarify the distinct contributions of the representation strategy and its optimisation.

AuthorsOpenReview ↗

Follow-up to Reviewer FXRR

Thank you again for your constructive feedback and suggestions. We have now uploaded a revised PDF with the changes highlighted.

In addition to the experiments discussed in our original response, we have extended the matched-budget search comparison to all three search LLMs. On 218 held-out K562 perturbations, using the same budget of 40 candidate evaluations per strategy, we obtain the following results.

Prediction performance:

Representation / strategyCDO ↑Corr@100 ↑MAE@100 ↓
Gene name0.1366 [0.1268, 0.1468]0.6713 [0.6430, 0.6990]0.5460 [0.5132, 0.5806]
NCBI/HGNC0.1475 [0.1374, 0.1577]0.7060 [0.6793, 0.7317]0.5282 [0.4954, 0.5633]
Canonical0.1546 [0.1450, 0.1645]0.7311 [0.7062, 0.7553]0.5095 [0.4764, 0.5448]
Qwen3.8-27B
Independent0.1540 [0.1444, 0.1639]0.7268 [0.7016, 0.7513]0.5114 [0.4782, 0.5466]
History-guided0.1777 [0.1645, 0.1916]0.7286 [0.6990, 0.7570]0.4423 [0.4119, 0.4745]
GPT-5.6 Sol
Independent0.1511 [0.1416, 0.1609]0.7252 [0.6996, 0.7499]0.5157 [0.4828, 0.5509]
History-guided0.1752 [0.1627, 0.1882]0.7256 [0.6961, 0.7535]0.4507 [0.4200, 0.4832]
Gemma-4-12B
Independent0.1547 [0.1451, 0.1648]0.7296 [0.7042, 0.7542]0.5091 [0.4759, 0.5444]
History-guided0.1806 [0.1675, 0.1943]0.7322 [0.7029, 0.7604]0.4370 [0.4065, 0.4693]

DEG recovery, strong-response overlap and perturbation discrimination:

Representation / strategyDEG AP ↑Signed Jaccard ↑PDS-L1 ↑PDS-cosine ↑
Gene name0.1271 [0.1070, 0.1483]0.0946 [0.0873, 0.1024]0.5754 [0.5357, 0.6142]0.5791 [0.5376, 0.6187]
NCBI/HGNC0.1331 [0.1126, 0.1547]0.1011 [0.0937, 0.1086]0.6014 [0.5617, 0.6409]0.6122 [0.5714, 0.6519]
Canonical0.1397 [0.1188, 0.1616]0.1056 [0.0987, 0.1130]0.6291 [0.5895, 0.6679]0.6445 [0.6054, 0.6835]
Qwen3.8-27B
Independent0.1393 [0.1184, 0.1612]0.1046 [0.0976, 0.1118]0.6236 [0.5839, 0.6626]0.6442 [0.6049, 0.6832]
History-guided0.1645 [0.1414, 0.1886]0.1376 [0.1252, 0.1504]0.7177 [0.6806, 0.7530]0.7290 [0.6892, 0.7672]
GPT-5.6 Sol
Independent0.1370 [0.1164, 0.1586]0.1031 [0.0963, 0.1102]0.6198 [0.5796, 0.6589]0.6347 [0.5952, 0.6741]
History-guided0.1595 [0.1371, 0.1829]0.1339 [0.1227, 0.1456]0.7075 [0.6702, 0.7433]0.7169 [0.6780, 0.7551]
Gemma-4-12B
Independent0.1401 [0.1192, 0.1621]0.1051 [0.0980, 0.1126]0.6292 [0.5893, 0.6679]0.6455 [0.6062, 0.6844]
History-guided0.1670 [0.1437, 0.1910]0.1421 [0.1297, 0.1551]0.7266 [0.6897, 0.7615]0.7347 [0.6958, 0.7720]

Values are means with 95% perturbation-bootstrap confidence intervals on 218 held-out K562 perturbations. All representations use matched Ridge predictors. Bold indicates the best point estimate.

History-guided optimisation improves held-out CDO relative to the expert-designed canonical protocol across all three search LLMs: Qwen3.8-27B (+15.0%; paired ΔCDO +0.0231 [0.0162, 0.0305]), GPT-5.6 Sol (+13.3%; +0.0206 [0.0131, 0.0279]) and Gemma-4-12B (+16.8%; +0.0259 [0.0186, 0.0336]). All three paired 95% confidence intervals exclude zero.

In contrast, independently generated proposals provide no improvement over canonical under the same evaluation budget. The benefits of history-guided optimisation also extend to MAE@100, DEG recovery, signed strong-response overlap and perturbation discrimination across all three search models, while Corr@100 remains largely unchanged.

The complete results and experimental qualifications are presented in Appendix H.7, alongside the expanded gene-embedding benchmark in Appendix H.8. We hope these additions help address your concerns.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.