acceptodds
Under review as a conference paper at ICLR 2027

Multi-Peptide Prompting Enables In-Context Learning in Protein Language Models

Abstract

We show that single-sequence PLMs can perform in-context peptide learning without gradient updates, task-specific retraining, or architectural modification. We introduce multi-peptide example prompts (MPEPs), in which demonstration peptides are concatenated with glycine spacers and used as context for scoring query peptides by their prompted probability. We evaluate this approach across multiple tasks, using both encoder-only ESM-2 models and decoder-only ProGen2 models. Across most tasks, performance improves with the number of peptide examples and with model scale, indicating that PLMs can extract shared properties from prompted examples. We further introduce a difference score that contrasts positive-example and negative-example MPEPs, reducing compositional biases in raw PLM probabilities and substantially improving classification. Perhaps surprisingly, when applied to few-shot peptide learning problems MPEP-based classification is competitive with classifiers trained on frozen ESM-2 embeddings, despite requiring no training. These results reveal an unexpected in-context inference capability in single-sequence PLMs and establish in-context learning as a potential lightweight strategy for low-data peptide classification.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
Protein language models, in-context learning

Reviews

Among:
Shape:
1234this paper, Bottom 25%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.75 over 4 reviews, above 25% of the others.
Reviewer_njzpOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper investigates in-context learning in protein language models using encoder-only ESM-2 and decoder-only ProGen2. It evaluates pattern completion, secondary-structure classification, and MHC-II binder classification, reporting performance comparable to a baseline using frozen ESM-2 embeddings.

Critical strengths

The paper addresses an interesting question: whether protein language models can perform in-context learning. The experiments and mathematical formulation appear reasonably thorough.

Critical weaknesses

The evaluation is limited to two model families and three classification tasks, providing insufficient model and task diversity to support the paper’s broad claims. Merely matching trained baselines also offers limited insight. Given the novelty and significance of the proposed capability, stronger evidence is needed to establish in-context learning in protein language models.

The absence of a related-work discussion, together with the disproportionate space devoted to standard scoring methods, raises concerns about the paper’s engagement with the existing literature.

Questions for authors
  1. Spacer choice. Why is G used as the spacer? Please cite relevant precedent or explain the rationale in the main text.
  2. Overstatement of the findings, around line 410. The statement “Our results provide strong evidence that single-sequence protein language models (PLMs) are capable of in-context peptide learning” is not adequately supported by two model families and three classification tasks. Please narrow the claim to match the evidence.
  3. Purpose of pattern completion. What scientific question does this experiment address, and how does it support the paper’s central argument? Its motivation and intended conclusion are currently unclear.
  4. Typographical error, line 211. Please correct “We next ask asks whether …”.
  5. Related work. Please discuss relevant prior studies and situate the contribution within the existing literature. The current omission raises concerns about whether the relevant research has been adequately considered.
1 comment

Request for clarification on evaluation

We would like to clarify the scope of the submitted evaluation. Firstly, in addition to the pattern completion, secondary structure classification, and MHC-II binder classification tasks, the original submission also evaluates our method across ace inhibition, antibacterial, anti-inflammatory, antimicrobial, antioxidant, neuropeptide, nonfouling, and tumor T-cell antigen tasks leveraging the PepBenchmark dataset (Main text: Section 3.5. Additional Experiments and Ablations; Appendix: Section D. PepBenchmark generalization and ablation studies, Figures 10-23). Additionally, we performed extensive mechanistic interpretability analyses on the ProGen2 model that yielded evidence for induction heads (Main text: Section 3.4 Induction Heads, Figure 6; Appendix: Section E. Induction heads in ProGen2 Table 3, Figures 24-66). These experiments were included before the submission deadline. Could the reviewer clarify whether the concern remains after accounting for these results?

1. Regarding the strength of the claim that protein language models (PLMs) are capable of in-context peptide learning

We dispute the reader’s claim around the strength of the claim. To show that PLMs are capable of ICL, it is sufficient to show that some are capable, and we are two-for-two. Our claim does not rest solely on a few matched benchmarks: our work on induction heads shows a clear mechanistic connection with in-context learning in LLMs.

2. Why glycine and whether the spacer is arbitrary

Reviewer concern: Explain the use of G as a spacer and provide a rationale or relevant precedent.

We address this in our general comment to the reviewers.

3. Purpose of pattern completion

Reviewer concern: What scientific question does the synthetic experiment address, and how does it support the central argument?

The experiment isolates a prerequisite for contextual adaptation: can additional copies of a sequence pattern change predictions at held-out positions? It separates this question from peptide-class labels and downstream biological classification. Increased completion accuracy with repeat count demonstrates sensitivity to in-context regularities, including artificial repeats over canonical amino acids.

4. Typographical correction

Reviewer concern: Correct “We next ask asks whether …” at line 211.

Thank you for catching this. The corrected wording is “We next ask whether …”. We will include this correction in the manuscript revision.

5. Related work and presentation

Reviewer concern: Add a substantive related-work discussion and rebalance space devoted to standard scoring methods.

We had incorporated discussion of related work inside the text, but have made it to a separate section in response to reviewer feedback. We are not clear on what is meant by “standard scoring methods”: for instance, whether the reviewer means the use of likelihood-based scores for PLMs or too much discussion of our benchmarks. We do not see the paper as having too long a discussion of benchmarks, although we could be persuaded otherwise. Regarding the discussion of scores for likelihoods, we are certainly not the first to sum over logits from a PLM! However, the use of the MPEP prompting strategy in PLMs is new. Consequently, even if the resulting likelihood may be obvious to a reader intimately knowledgeable of the field, it nevertheless needs to be stated precisely to avoid ambiguity.

Reviewer_RVbiOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper investigates whether pretrained single-sequence protein language models can support few-shot peptide classification through prompting alone. The proposed multi-peptide example prompt concatenates demonstration peptides using five-glycine spacers. Queries are scored using their prompted log-probabilities, with a contrastive variant subtracting scores obtained from negative-example prompts from those obtained from positive-example prompts.

For ESM-2, masking the entire query produces a reusable position-specific scoring matrix; ProGen2 instead scores queries autoregressively. Experiments cover synthetic pattern completion, secondary-structure classification, MHC-II binding for one allele, and eight PepBenchmark tasks partitioned into 41 length-specific bins. The authors report improvements with additional demonstrations, benefits from contrastive scoring, and MHC-II performance approaching classifiers trained on frozen ESM-2 embeddings. Additional controls examine composition, example ordering, and the contribution of induction heads in ProGen2.

Critical strengths
  1. A simple and potentially useful adaptation of existing PLMs. The method requires neither parameter updates nor a specially trained multi-sequence architecture. Its strongest practical feature is that ESM-2 generates a scoring table that can be reused across queries of the same length. Table 1 provides encouraging evidence: with 32 examples per class, ESM-2 650M reaches ROC-AUC 0.86 ± 0.02, compared with 0.77 ± 0.02 for the reported contrastive PSSM and 0.88 ± 0.01 for embedding-based logistic regression. This suggests a potentially attractive accuracy–computation tradeoff.

  2. The supplementary controls substantially strengthen the empirical observation. The authors examine random background prompts, label mixing, residue scrambling, and position-specific versus composition-based synthetic demonstrations. These controls help establish that the improvement depends on class-relevant context. The ProGen2 experiments also connect identified induction heads to downstream classification: Table 3 reports an xlarge-model ROC-AUC decrease from 0.866 to 0.799 after induction-head ablation, while the layer-matched control remains at 0.866. This provides useful mechanistic evidence beyond attention visualizations alone.

Critical weaknesses
  1. The advance over existing knowledge is narrower than the overarching framing suggests. Kantroo et al. (2025b) already demonstrated context-dependent retrieval in single-sequence ESM2 and ProGen2, including on random sequences and imperfect repeats. Thus, context responsiveness in these models is already established. Conditioning protein predictions on example sequences also has substantial precedent, although approaches such as PoET and Metalic use dedicated architectures or additional training.

    The distinctive contribution here is therefore the usefulness of a particular training-free, class-conditional peptide scorer. For a fixed prompt and query length, the ESM-2 difference score is exactly

    W_i(a)=\log p_i(a\mid C^+)-\log p_i(a\mid C^-).$$ This is a position-additive classifier whose weights are inferred from demonstrations by a pretrained model. That is a valid and interesting form of context adaptation, but its output has the expressive power of a PSSM. The paper acknowledges this restriction; nevertheless, the broader interpretations about inferring shared biological properties and emergence independently of training data exceed what is established. The central scientific question is how much useful transfer the pretrained model contributes to estimating these profiles beyond well-regularized statistical estimators.
  2. The comparative evaluation does not yet establish that central added value broadly enough. The main comparison against PSSM, one-hot classifiers, and embedding-based classifiers is restricted to one MHC-II allele, using the highest- and lowest-affinity groups after discarding intermediate examples. The PepBenchmark extension adds breadth, but compares prompting variants without extending these competing classifiers to the same tasks. Consequently, it establishes that prompting produces predictive signal across tasks, while leaving its comparative advantage largely unresolved.

    The PepBenchmark protocol also reports exact deduplication followed by random splitting, without quantifying residual train–test sequence similarity. Its main heatmaps average only three prompt draws without uncertainty intervals. Although the main structural and MHC-II datasets include redundancy reduction, stronger characterization of the supplementary splits and paired baseline comparisons is needed to assess generalization. These gaps matter because the paper’s main advance over previously observed context effects is its proposed utility as a classifier.

Questions for authors
  1. Comparative evidence: On the existing PepBenchmark bins, how does MPEP compare with a smoothed or regularized PSSM, a composition-based classifier, and frozen-ESM embedding logistic regression, using identical demonstration sets and validation budgets? Please specify the PSSM pseudocounts and baseline regularization, and report uncertainty for paired performance differences. These comparisons would materially affect my assessment of the method’s added value.

  2. Evaluation protocol: Please specify the disjoint training, validation, and test counts for the MHC-II experiment and explain the difference between Figure 4 and Table 1 at small k. For example, ESM-2 650M difference scoring appears near 0.50 at k = 1 in Figure 4A, whereas Table 1 reports 0.65 ± 0.13. Were different splits or sampling procedures used? For PepBenchmark, what is the distribution of each test peptide’s maximum sequence identity to the training pool?

  3. Efficiency claim: What hardware, batch sizes, peptide lengths, and timing boundaries underlie Figure 5? Does MPEP timing include constructing both positive and negative scoring tables, and do embedding baselines use batched inference? Please distinguish scoring a new candidate library from reusing previously computed embeddings. This clarification is necessary to interpret the claimed three-orders-of-magnitude speedup.

1 comment

We appreciate the reviewer’s positive comments on our models’ simplicity and use. Regarding the other points brought by the reviewer, we have sought to clarify the novelty of the method, and have also provided additional benchmarks to provide further variability.

1. Novelty and the expressive power of the scorer

Reviewer concern: Context-dependent retrieval is already known, and the fixed-prompt ESM-2 score has the expressive power of a PSSM. Claims about shared biological properties and emergence independently of training data are too broad.

We agree with both of these statements. The work by Kantroo et al. showed that context-dependent retrieval is possible, and that fixed-prompt has the expressive power of a PSSM. However, we do not believe this restricts the novelty nor the importance of our work.

Kantroo et al. primarily focused on how context-dependent retrieval causes a sequence with low fitness to have unusually high likelihoods when that sequence contains repeated motifs. However, it was not obvious prior to our work that PLMs like ESM-2 or ProGen2 can productively utilize context to predict sequence dependent properties from limited data.

Regarding the expressive power of PSSMs, we agree with the statement. However, we also don’t see this as a practical limitation of the method. Expressive power matters insofar as a learning task has the data to support it. In fact, a primary motivation for us pursuing this work was repeated discussions with experimentalists who came to us with “machine learning” tasks with tens of datapoints. Moreover, strategies such as autoregressive decoding, while slower, are in theory more expressive.

Regarding claims about shared biological properties, we do not think they are too broad, rather they are a simple statement that we achieve a nontrivial classifier. Similarly, regarding our statement about emergence independent of training data we simply intend to establish that we have seen productive in-context learning emerge in the absence of language data. In either case, we will reword our statements to avoid any unintentional confusion with greater claims.

2. Comparative evidence and paired variability

Reviewer concern: Extend matched PepBenchmark comparisons to regularized PSSMs, composition classifiers, and embedding logistic regression; specify regularization and report uncertainty in paired differences.

This point is well taken: we were unable to prepare the PepBenchmark numbers in time for our initial submission. Following this feedback and from other reviewers, we have extended our results to PepBenchmark. We have detailed our experiments in the general comment.

3. Data splits and the Figure 4 discrepancy

Reviewer concern: Give the disjoint MHC-II counts and explain why small-k results differ between Figure 4 and Table 1. Report PepBenchmark test-to-training sequence similarity.

Thank you for pointing that out. For small-k results in figure 4, we did not use the held-out validation set to choose the optimum set of peptide examples, whereas we did perform this hyperparameter search for the data in table 1.

Reviewer_U5LZOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes a novel way of implementing ICL in Protein Language Models (PLMs) to classify peptides. The authors suggest that ICL is a difficult problem in PLMs because they are "never explicitly trained to compare multiple sequences or aggregate information across demonstrations". However, they demosntrate that this can work using glycerine spacers. They concatenate multiple peptides into a multi-peptide example prompt (MPEP), and pass it into a pretrained PLM "without gradient updates, parameter modification, or task-specific retraining". They use log-probs to classify whether the sequence is likely. They implement this for two PLM families: encoder-only, and decoder-only, with a slightly different approach on how log-probs are summed. The authors conduct extensive experiments, and ablations, including a demonstration count ablation and an induction head study.

Critical strengths
  1. ICL in PLMs appears to be an important question, which has not been resolved before. Understanding whether ICL applies to PLMs presents useful intuition on their generalization, as well as provide an interesting use case to how PLMs can be used.
  2. Increasing the number of demonstrations improves the accuracy, which is consistent with ICL. This suggests additional examples can provide higher performance.
  3. The authors conduct extensive experimentation and ablations across multiple PLM architectures, tasks, and PLM model sizes.
  4. The ICL approach is quite interesting. Instead of generalizing on input, the generalization is based on the propoerty shared by the the demonstration and the test peptide. They present an induction head experiment that provides additional evidence that the behavior is in fact ICL.
Critical weaknesses
  1. This paper is targeted to a specialized audience, and provides insufficient background to help readers who are not familiar with the subtleties of PLMs, such as how they are trained (even in a short paragraph), and what is the expected inputs and outputs.

  2. The research gap of why ICL in the traditional sense would not work with PLMs is insufficient motivated, and neither is the previous work attempting this. In fact, the related work section is completely missing from the manuscript. As a result, the contribution of this paper is not well positioned within the relevant literature.

  3. The authors insufficiently motivate the spacer, why it was selected, and why it helps make ICL possible in PLMs. Would other spacers perform equally well? Were any pilot studies conducted to help motivate the choices made?

Questions for authors
  1. How does ICL improve on the zero-shot performance of PLMs on all tasks? What tasks are most improved by the use of ICL?
  2. What is the motivation for the selected spacer, and would other spacers work equally well?
  3. Would you consider this to be a form of self-supervised ICL?
2 comments

We appreciate the reviewer’s recognition of the importance of ICL in PLMs, as well as their queries about We address the background and positioning concerns first, followed by your specific questions.

1. Background and the research gap

Reviewer concern: Explain how PLMs are trained, their inputs and outputs, why conventional ICL is not directly applicable, and the relevant prior work.

We will add a short introduction to the two model families, as well as a discussion of how protein language models (PLMs) are trained and their inputs and outputs. To answer the reviewer’s question in brief, the core issue is that PLMs – unlike LLMs – have no access to text. Their inputs are amino-acid tokens and their native outputs are token probabilities, rather than natural-language instructions or peptide-class labels. Consequently, normal ICL doesn’t work because many PLMs have only ever seen single protein sequences. In fact, our use of multiple peptide sequences in these models is already, to our knowledge, novel.

We addressed related work in the body of the text. We agree that moving it into a separate section would make the paper more in-keeping with the typical ICLR style, and have adjusted the text accordingly.

2. Spacer motivation and alternative choices

Reviewer concern: Why was GGGGG selected, were pilot studies used to justify it, and would other spacers perform equally well?

GGGGG was a heuristic separator, not an empirically established optimum. It was chosen primarily for simplicity and ease of presentation. Following reviewer feedback, we have evaluated a larger suite of spacers, and we refer the reviewer to the general comment above for details of our experiments.

3. Improvement over zero example performance

Reviewer concern: How does ICL improve on zero-shot performance across tasks, and which tasks benefit most?

We're not sure exactly what you mean by this question. Given the context of the rest of your comment, we think you are asking about zero-shot ICL in the traditional natural language sense. Here is our response to that interpretation of your question. Feel free to provide additional clarification.

Unlike a natural language model, PLMs can only accept amino acid tokens as input and as such have no direct way to write task specific instructions or descriptions typically employed by zero-shot ICL – each example is implicitly labeled by building an entire prompt of peptides that share a label. Therefore, tasks must be encoded implicitly. With no examples, the model just gives the unconditional sequence preference and is completely unaware of the classification task.

4. Whether the method is self supervised ICL

Reviewer concern: Would this be considered a form of self-supervised ICL?

Our method is not a form of self-supervised ICL. However, the use of MPEP as part of a self-supervised ICL pipeline – particularly given its low inference cost – poses a promising future direction.

I thank the authors for their responses. However, the lack of related work, along with some of the points in the authors' general comment suggests the work may still be at a relatively early stage. As such, I will maintain my current rating.

Reviewer_Z5PtOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The authors investigate whether pretrained, single-sequence protein language models can classify peptides using examples provided in the context, without updating model parameters (by passing peptides concatenated and linked with 5-glycine spacers). A difference between scores from positive and negative example prompts improves classification. They evaluate ESM2 and Progen2 on synthetic pattern completion, secondary-structure classification, and MHC-II binding. Validation includes comparisons with PSSMs and classifiers trained on ESM embeddings, composition and example-selection controls, and induction-head ablations in ProGen2.

Critical strengths

The method provides a simple way to obtain useful peptide classification scores without updating the pretrained model. On the MHC-II task, performance is close to classifiers trained on frozen ESM embeddings.

The authors go beyond reporting classification performance and investigate the context effect through several controls. In particular, the ablations with ProGen2 provide mechanistic insights.

Critical weaknesses

The comparative evaluation is too restricted to establish general practical usefulness. The main comparison against ESM embedding classifiers uses one MHC-II allele and selects the strongest and weakest binders, discarding the intermediate measurements. The supplementary PepBenchmark experiments extend the task coverage, but do not provide equally broad comparisons with embedding classifiers. Broader comparisons under a clearly controlled sequence-similarity split would be needed to establish when this approach offers an advantage over existing methods.

I am not convinced that the approach offers a sufficiently clear advantage over existing methods. In Table 1, classifiers trained on frozen ESM embeddings achieve the highest mean AUROC in five of the six example-count settings. The proposed method therefore generally matches or underperforms standard alternatives. Although the supplementary PepBenchmark experiments broaden the task coverage, they do not provide equally broad comparisons against embedding classifiers. The computational savings could be useful, but the current evaluation does not sufficiently establish when these justify choosing this over existing approaches.

I also find the prompt construction not a particularly elegant approach. The method consists of concatenating peptides with five glycine spacers, but the choice of separator and its length are not investigated. Why glycines? It remains unclear how much the results depend on this particular input format. The experiments are also restricted to fixed-length peptides, leaving broader applicability insufficiently established. Since the prompt format is the main methodological contribution, I would expect stronger justification of its design and a more thorough assessment of its robustness.

Questions for authors

None

2 comments

We appreciate your comments, as well as your concerns about the restrictiveness of the analysis. We have endeavored to fix this by adding more extensive benchmarks, as well as opening spacer design as a new direction of inquiry.

1. Broader comparisons and practical advantage

Reviewer concern: The MHC-II evaluation uses extreme-affinity examples from one allele, embedding classifiers usually perform best, and the supplementary tasks lack equally broad baseline comparisons.

To put our results on firmer footing, we have completed the PepBenchmark baselines to allow for fair comparison: we ran out of time to complete these for our initial submission. Broadly speaking, our results mirror those for MHC-II: the MPEP-based ICL performs comparably to embedding-based classifiers at significantly lower computational cost. We stress that we do not believe the value of our work rests in outperforming frozen-embedding baselines. Rather, our contribution is twofold. First, we make a theoretical claim, showing that ICL is more general than previously known, and is innate to PLMs. Second, we show that ICL is also a practical tool, achieving results that are comparable to other lightweight classifiers, but being faster to run. This is of real practical benefit for experimental workflows, which need to scan through combinatorially large sequence spaces.

2. Spacer design and robustness

Reviewer concern: Why use five glycines, and how sensitive is the result to spacer sequence and length?

This was a common question by the reviewers; consequently, we addressed this in a general comment above.

3. Peptide length and context limits

Reviewer concern: The experiments use fixed-length peptides, limiting broader applicability.

Our primary objective is to show that ICL behavior exists in protein language models trained on single sequences and can be used for classification tasks in the low-data limit. We view variable length peptide examples as an interesting direction for future work.

Thank you for your comments. I still consider it a rejection. The prompt construction does not constitute a particularly novel approach. The text reads rushed and requires more experiments. At this stage, the work reads more appropriately for a workshop venue rather than a main conference.

Comments on the paper

1 comment

Opening Statement

We thank the reviewers for their careful evaluation. Their comments identify three issues that need clearer treatment: the practical value relative to established classifiers, the dependence on spacer design, and the scope of the in-context learning claim. First, we have completed frozen-embedding benchmarks for the pepbenchmark task: we were unable to complete these prior to the initial submission deadline due to time constraints, and we apologize for this omission. Secondly, in response to viewer feedback we have completed comparisons of 32 handpicked spacers on PepBenchmark and MHC-II tasks. These results support a useful but conditional benefit: spacer selection improves PepBenchmark averages across all four ESM-2 sizes tested, while its benefit on the MHC-II task depends on model size and example count, but is generally positive. We are currently finishing editing the draft to include these changes. However, due to their extent the rewrites are still in progress. We will update the draft shortly; below we address each reviewer’s concerns and specify the manuscript clarifications supported by these results. Reviewer concerns are paraphrased for brevity.

Why choose GGGGG as a spacer?

Our choice of polyglycine spacers was motivated by prior work that adapted single-sequence protein folding models including AlphaFold2 and ESMFold for multichain complex prediction by concatenating protein sequences with polyglycine linkers [1, 2]. We adopted the same input-construction strategy to combine multiple peptide examples into a single prompt, using polyglycine spacers to separate adjacent sequences.We chose GGGGG as a simple initial spacer made from the model’s ordinary amino-acid vocabulary, rather than an optimized sequence. As the primary focus of our paper was establishing phenomenology rather than tuning performance, we opted for simplicity in our initial presentation. However, we have taken the Reviewers’ feedback that the choice of spacer requires more investigation to heart.

Consequently, we introduced a new panel of handcrafted spacers mined from the literature that contains 32 amino acid-based spacers spanning 3–17 residues, including glycine/serine variants, proline-containing sequences, and EAAAK-based helical designs. We assessed the performance of the handcrafted spacers on the PepBenchmark and MHC-II tasks. Twenty candidates are five residues long, allowing length-matched comparisons with GGGGG.

Table R1. MHC-II test ROC-AUC at 32 training peptides per class. The last column is the mean ± sample SD of within-split handpicked-minus-GGGGG differences across ten splits.

ESM-2GGGGGHandcraftedPaired change
8M0.842 ± 0.0290.844 ± 0.025+0.0016 ± 0.0267
35M0.862 ± 0.0200.853 ± 0.027-0.0096 ± 0.0271
150M0.863 ± 0.0180.875 ± 0.018+0.0128 ± 0.0199
650M0.861 ± 0.0230.879 ± 0.012+0.0186 ± 0.0165

These results show that useful classification is possible with alternatives to GGGGG. Interestingly, we do not see one universally best spacer, suggesting that spacer choice is task and data specific, and should be tuned as a hyperparameter during validation. We will explicitly state this in the paper.

Additional Results on PepBenchmark

While we started the PepBenchmark results, we were unable to provide the frozen embedding-based classifier reference data prior to the initial submission. We have since completed these analyses, and are providing them in the Supplementary Information. To summarize our results, we see that MPEP-based classifiers with handcrafted spacers perform better than embedding-based classifiers for two ESM-2 models (35M and 150M) and are within 2 percentage points of the logistic regression and 3-layer MLP models trained on ESM-2 650M embedded sequences when using handcrafted sequences.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.