acceptodds
Withdrawn from ICLR 2027

Evolutionary Informed Translation from Protein to Nucleic Acid Sequence

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

The choice of nucleic acid sequence for a given protein strongly impacts function and is species-specific. Current protein-to-nucleic-acid translation models treat sequences in isolation, often over-favoring frequently used codons. To generate more natural nucleic acid sequences, we introduce PINATA (Phylogenetically Informed protein to Nucleic Acid Translation with sequence Alignment), a model that leverages sequences from related species encoded in multiple sequence alignments (MSAs) to guide prediction. Our architecture leverages evolutionary context and learns species-and protein specific context to make biologically-informed predictions. Evaluated across 61,934 gene trees and 882 species, PINATA generates sequences that mirror the naturally-occurring sequence substantially better than existing language models. PINATA leverages MSA data to improve exact native codon recovery by 17.95 percentage points over existing language models, effectively preserves natural rare-codon nuances, and more closely matches native sequence properties. PINATA maintains strong predictive performance even with limited evolutionary data. While previous methods largely treated sequences from different species in isolation, our method leverages the evolutionary relationships between nucleic acids and proteins across species to create a more biologically-informed translation. We demonstrate that leveraging rich evolutionary data through an biologically-designed method strongly aids in translating protein sequences to nucleic acid sequences across species. We will make data and code publicly available upon publication.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
natural language processing, computational biology

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_R5jqOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper studies the translation problem from protein sequences to their corresponding codon sequences given a particular species. Specifically, this paper proposes PINATA, which leverages MSAs, encoded into a hand-engineered 34-dimensional matrix from related species to guide prediction. PINATA is trained in a similar way as masked language model with additional supervision on matching the properties of native species. In evaluation, PINATA is shown to predict codon sequences that match the properties of native species better than existing baselines.

Critical strengths

The paper studies an important field that connects protein sequences with codon sequences, which has impactful applications in drug discovery. The paper explicitly takes MSA into the modeling which is novel.

Critical weaknesses

I'm not convinced by the evaluation because the evaluation metrics are part of the training loss of PINATA. Therefore, it is not surprised that the PINATA predict codon sequences that match the properties of native species and it is unable to tell if the improved performance is from the direct signal of supervised training or adding evolutionary information from MSA.

Questions for authors
  1. The evaluation metrics are the same as part of the loss function. Could the authors conduct another ablation study to remove those property loss and only keep the reconstruction loss during training?

  2. The paper extracts a 34-dimensional matrix from MSA. Why don't apply another transformer on the raw MSA directly without using hand-crafted features? Moreover, the similarity between species are not taken into account in the MSA statistics, which are important evolutionary information. The performance of learning-free MSA is only slightly worse than PINATA, which indicates that modeling raw MSA directly might be more effective.

  3. What downstream applications that PINATA can do beyond predict codon sequences that match the properties of native species?

Reviewer_p9ueOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

Summary: Multiple codons can encode the same amino acid, and codon usage varies across species. PINATA uses multiple sequence alignments (MSAs) of homologous genes to improve DNA sequence prediction from proteins. It extracts 34 features per position, including codon usage across homologs and species-specific codon preferences, and adds them to the input embeddings of a 90M-parameter BERT-style model. Training uses masked language modeling, followed by an additional epoch with losses for GC content and codon usage statistics.

Contributions: The paper proposes using position-wise information from homologous sequences to improve codon prediction. It reports a 17.95 percentage-point improvement over the strongest baseline.

Validation: The dataset includes 5.93M coding sequences from 882 species, with protein-clustered splits repeated three times. PINATA achieves 67.94% codon accuracy, compared to 49.99% for CodonTransformer and 43.21% for SynCodonLM. Removing MSA features reduces accuracy to 50.35%, while a simple majority-vote heuristic achieves 63.01%. Performance also decreases as close homologs are removed. On an unseen-species bacterial test, PINATA achieves 48.13%, compared to 43.09% and 44.60% for the baselines. The evaluation is computational, based on similarity to natural sequences.

Critical strengths
  1. The splits are rigorous. Proteins are clustered with MMseqs2 at 80% identity and 80% coverage, and each cluster goes entirely into one split. Results are averaged over three splits with reported variance, and there is a within-tree control set.

  2. The ablations are honest. The authors report the simple heuristic that accounts for most of their gain. They also report that their model does worse than its own no-MSA version once the MSA is removed.

Critical weaknesses
  1. The method has limited novelty, and the learned model adds little. Adding MSA-derived features to input embeddings is a standard technique. A simple majority vote over homolog codons already achieves 63.01%, compared to PINATA’s 67.94% and CodonTransformer’s 49.99%. On bacteria, PINATA improves over the heuristic by only 0.74 percentage points. Overall, the method is not particularly new, and the additional gains from learning are small.

  2. The evaluation favors PINATA. Four of the six evaluation metrics are already included in PINATA's training loss, but not in the baselines' losses. There is no ablation to assess this advantage. PINATA also receives extra training beyond the claimed five-epoch budget, including a fully masked final epoch matching its evaluation setting. It is therefore unclear how much of the improvement comes from the model itself.

Questions for authors
  1. What does PINATA score without the property loss, on all six metrics?
  2. How many optimizer steps did each model get, and how much comes from the full-mask epoch?
  3. Why do the baselines miss the signal that your majority-vote heuristic captures?
  4. What is the performance with only the closest k homologs (k = 1, 2, 5, 10)?
  5. Can you break the results down by division and by species abundance?
Reviewer_jVofOpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The goal is to generate more natural codon sequences for a given protein; to do so, the authors propose to leverage MSAs from related species. Previous methods condition on the species without leveraging evolutionary relationships between species. Here, the authors propose a method (PINATA) that injects evolutionary context from MSAs of homologs into a codon LM. Specifically, in the authors' experiments, PINATA leverages MSA data to improve exact native codon recovery by 18 percentage points over existing codon LMs.

Critical strengths

To my knowledge, this paper is the first to leverage MSAs for codon sequence prediction. Also, their evaluation spans >61,000 gene trees and >800 species across five taxonomies, with two test settings to separate generalization from having seen homologs during training.

Critical weaknesses
  1. Evaluation metrics include GC content, Codon Frequency Deviation (CFD), Codon Similarity Index (CSI), etc. These are terms directly optimized in PINATA's loss Lproperty\mathcal{L}_{\mathrm{property}}, which explains why PINATA performs so well under these metrics in e.g. Tables 2-3. I'm interested in how well PINATA performs without these bespoke loss terms (or how the baselines perform with them); do these loss terms actually improve other quantities of interest, like exact native codon recovery?
  2. In general, I'm not yet convinced by the results due to insufficient baselines and ablations. For example, PINATA cites a few papers that use the most closely aligned homolog as context; PINATA introduces more homologs as context, so the authors should demonstrate using more homologs significantly improves performance. It would be nice to have single-homolog baselines like OrthologTransformer and CodonT5. Table 4 reports performance with reduced MSA context, but the authors drop the closest homologs in the context first, so to my understanding they don't have my requested single-homolog baseline. Also, see Questions 2-3 below.
Questions for authors
  1. I'm not sure, but is it obvious how ΔGC\Delta GC and other terms in Lproperty\mathcal{L}_{\mathrm{property}} are differentiable?
  2. I'm curious how much the transformer architecture contributes, can the authors train a simple network or even multinomial classifier on the MSA context vector and report results for that?
  3. Can we compare against some simple baselines, like (a) predicting the codon with highest πs,c\pi_{s, c} weight per-residue; (b) sampling the codon from πs,c\pi_{s, c} per-residue; (c) predicting the codon of the position-matched nearest homolog (falling back to (a) or (b) at positions with no matching homolog).

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.