acceptodds
Under review as a conference paper at ICLR 2027

Prot2RNA: Reward-Guided Discrete Diffusion for Protein-Conditioned CDS Design

Abstract

A single protein can be encoded by many synonymous mRNA coding sequences (CDSs), whose composition affects RNA structure, stability, and protein output. We introduce **Prot2RNA**, a protein-conditioned discrete diffusion model that learns a conditional distribution over natural human CDSs. Motivated by the role of RNA structure in protein output, we shift this conditional distribution toward sequences with increased predicted structural stability using normalized minimum free energy (MFE) as a non-differentiable sequence-level reward, through reinforcement finetuning across diverse proteins (**Prot2RNA-RFT**), followed by test-time reinforcement learning for a fixed target (**Prot2RNA-TTRL**). Prot2RNA-RFT consistently shifts normalized MFE in the intended direction across held-out proteins, while Prot2RNA-TTRL further adapts the policy for a single NanoLuc protein. In HEK293T cells, the selected NanoLuc design achieves the highest measured reporter output among the tested constructs, with 39% and 71% higher output than GEMORNA and LinearDesign, respectively. These results demonstrate that a learned protein-conditioned CDS prior can be effectively optimized toward a sequence-level reward using reinforcement learning, and when adapted to a single target, yield experimentally validated designs that outperform established CDS-design methods.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
diffusion language model, mRNA design, reward alignment, protein-conditioned generation

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_L21mOpenReview ↗
Rating
1(−1)
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper introduces Prot2RNA, a pipeline consisting of a discrete diffusion model trained for mRNA CDS generation, RL finetuning with GRPO to maximize minimum free energy (MFE), and test-time reinforcement learning with fixed protein targets. The method is validated in silico and in vitro in HEK293T cells, showing higher reporter output compared to published baselines.

Critical strengths
  • The empirical results are solid with some wet-lab validation, showing improvements against published baselines like GEMORNA.
Critical weaknesses
  • There is no clear theoretical or algorithmic novelty, and the algorithm seems to be a simple combination of discrete diffusion and RL finetuning with GRPO. It also doesn’t cite or compare against several relevant baselines, such as [1] or [2]. My current stance on this paper is that it is unsuitable for publication at an ML conference like ICLR.
  • Validation was performed on NanoLuc only and no other targets.

[1] Zekri, O., & Boullé, N. (2025). Fine-tuning discrete diffusion models with policy gradient methods.

[2] Tang, S., Zhu, Y., Tao, M., & Chatterjee, P. (2025). Tr2-d2: Tree search guided trajectory-aware fine-tuning for discrete diffusion.

Questions for authors
  • The normalized MFE and NanoLuc output seem weakly correlated in Figure 5, is there a way to distinguish the reason for the gain in NanoLuc output in Prot2RNA over the designs from LinearDesign, which has the highest normalized MFE?
  • What is the total number of reward evaluations used to design the best TTRL sequence, and how does it compare to best-of-N at an equal budget?
  • Can the authors show improved performance for more targets other than NanoLuc, at least in silico, with TTRL?
4 comments
AuthorsOpenReview ↗

Reply to Reviewer L21m

We thank Reviewer L21m for taking the time to review our manuscript and for their constructive feedback. We sincerely appreciate their recognition of our strong empirical improvements over existing baselines such as GEMORNA, as well as the value of our wet-lab validation in providing experimental support beyond computational metrics that are often lacking in the related work.

Response to weaknesses comments:

We thank the reviewer for highlighting SEPO [1] and TR2-D2 [2] and we are happy to include both in the related work. These methods propose alternative reinforcement fine-tuning algorithms for discrete diffusion, evaluated on tasks such as regulatory DNA and peptide design, rather than protein-conditioned CDS generation. A controlled quantitative comparison would require adapting their optimization algorithms to our protein-conditioned, synonymously constrained diffusion model. Our work does not claim algorithmic superiority over alternative RL methods, nor do we claim superiority of GRPO against other policy optimization methods; instead, it investigates the use of discrete diffusion for CDS generation and its adaptation to sequence-level rewards, supported by computational comparisons with established CDS-design methods and preliminary experimental validation. While we agree it would be interesting to compare RL methods such as SEPO, TR2-D2 and GRPO, we believe it is out of scope for this paper.

Answering the Reviewer L21m questions:

Question 1

Normalized MFE is not a direct predictor of protein output, as discussed in Sections 3.4 and 4. For example, in the GEMORNA paper [3], we can also see that GEMORNA outperformed LinearDesign [4] in experimental expression of firefly luciferase and COVID-19 spike, despite substantially lower normalized MFE values (0.419 vs. 0.670 and 0.370 vs. 0.601, respectively). Protein output depends on multiple interacting factors, including transcript stability, translation dynamics, and translation-dependent decay. Although the LinearDesign construct achieves the highest normalized MFE, this does not necessarily translate into higher reporter output. The hypothesis was that Prot2RNA preserves favorable sequence properties learned from natural CDSs while optimizing structural stability. In silico metrics are not directly transferable to the experimental total protein output. Wet-lab experiments, which are expensive and time-consuming, are the only proper way to test the effectiveness of the mRNA design.

Question 2

Our original best-of-N comparison used N = 800, matching the reward-evaluation budget of a single TTRL configuration (50 optimization steps, with 16 generated sequences per step). This comparison was intended to assess whether updating the generative policy provides an advantage over additional sampling from the frozen RFT policy.

For NanoLuc, we performed a staged hyperparameter search because suitable test-time adaptation settings were not known a priori. We selected hyperparameters based on the mean reward of generated sequences at the final training step, reflecting the adapted policy's performance rather than a single high-reward sample. We then selected the highest-reward CDS encountered during the chosen configuration's training trajectory for experimental evaluation.

We agree that the full hyperparameter-search budget should also be considered. Across the 51 configurations explored, the total target-specific cost was 49,200 reward evaluations. We therefore extended best-of-N sampling from the frozen RFT policy to N = 49,200, matching the full NanoLuc TTRL search budget. The expected best normalized MFE increases from 0.4733 at N = 800 to only 0.4785 at N = 49,200, compared with 0.4977 for the TTRL-selected sequence and a mean of 0.484 across 64 samples from the selected TTRL policy.

Thus, even with the full target-specific evaluation budget, additional sampling from the frozen RFT policy does not reach the reward achieved through TTRL. This demonstrates that target-specific policy adaptation can access higher-reward designs than those found by increasing the sampling budget of the original RFT policy.

Question 3

We extended our in silico TTRL evaluation to two additional targets: an eGFP-based reporter (284 aa) from the Moderna study [5] and an engineered erythropoietin (EPO) construct (221 aa) from GEMORNA [3]. Neither target, nor closely related protein sequences, was represented in the SFT or global RFT training sets.

Starting from Prot2RNA-RFT, we performed target-specific reinforcement learning using the same normalized-MFE reward as for NanoLuc. To distinguish improvements through policy adaptation from increased sampling of the starting RFT policy, we compared TTRL against best-of-N selection from the frozen RFT policy at a matched budget of 800 reward evaluations per run.

Normalized MFE (higher is better)eGFPEPO
RFT best-of-8000.45050.5509
TTRL policy mean0.46010.5996
TTRL best during search0.46330.6056
RNARL [6]0.39930.5240
GEMORNA0.33290.4733
LinearDesign (λ = 3.3)0.57610.6817

For both targets, ordinary stochastic sampling from the selected TTRL policy achieved higher mean normalized MFE than best-of-800 sampling from the starting RFT policy. The highest-reward sequences identified during TTRL reached 0.4633 for eGFP and 0.6056 for EPO, demonstrating additional reward improvement through target-specific policy adaptation.

For context, the highest normalized MFE among 30 published Moderna eGFP constructs was 0.4095, while the highest among five published GEMORNA EPO designs was 0.4913. TTRL also exceeded the RNARL [6] and GEMORNA-generated designs on this objective, whereas LinearDesign achieved higher normalized MFE, consistent with its explicit optimization of RNA folding stability. As discussed in our NanoLuc experiments, however, higher normalized MFE does not necessarily translate into greater protein output.

Together with NanoLuc, these results demonstrate that TTRL can further shift a reusable RFT policy toward higher reward across multiple protein targets, beyond what is achieved through additional sampling from the frozen policy. Experimental evaluation on these additional targets remains an important direction for future work.

References:

[1] Zekri, O., and N. Boullé. "Fine-tuning discrete diffusion models with policy gradient methods." Advances in Neural Information Processing Systems 38 (2026): 152868–152906.

[2] Tang, S., et al. "TR2-D2: Tree search guided trajectory-aware fine-tuning for discrete diffusion." arXiv preprint (2025).

[3] Zhang, He, et al. "Deep generative models design mRNA sequences with enhanced translational capacity and stability." Science 390.6773 (2025): eadr8470.

[4] Zhang, He, et al. "Algorithm for optimized mRNA design improves stability and immunogenicity." Nature 621.7978 (2023): 396-403.

[5] Bicknell, Alicia A., et al. "Attenuating ribosome load improves protein output from mRNA by limiting translation-dependent mRNA decay." Cell Reports 43.4 (2024).

[6] Lin, Shenggeng, et al. "RNARL: reinforcement learning-driven unified generative framework for multi-objective RNA codon design." Genome Biology (2026).

Thank you for the rebuttal. However, my main concerns regarding theoretical novelty and limited evaluation remain. I argue that the suggested baselines can be easily adapted to protein-conditioned CDS generation by swapping out the rewards and finetuning the same base model. In addition, it seems like the added results on eGFP and EPO do not outperform LinearDesign. Given these limitations, I recommend reject.

AuthorsOpenReview ↗

Reply to Reviewer L21m

Dear Reviewer L21m,

We appreciate your suggestions and would like to clarify the contribution and the purpose of our comparisons. Our contribution is a framework for adapting a protein-conditioned generative model to mRNA design objectives; we do not propose a new reinforcement learning algorithm. GRPO serves as the optimization method for adapting the model to sequence-level rewards. Accordingly, GEMORNA, CodonBERT, Trias, and CodonTransformer provide application-level comparisons, while comparisons against the starting and frozen policies assess the benefit of adaptation. Evaluating GRPO against SEPO or TR2-D2 would address a different question—which policy optimization algorithm performs best within this framework—and we make no claim of superiority on that question.

We agree that discrete diffusion and GRPO are established methods. The contribution lies in how these components address mRNA sequence design and in the evidence supporting the resulting framework. We therefore ask that the work be assessed against its application objective and empirical findings. This is consistent with the ICLR Reviewer Guide, which explicitly recognizes better addressing a known application or problem as a valid research objective and states that different objectives require different considerations of value and impact. https://www.iclr.cc/Conferences/2027/ReviewerGuidelines

For both additional protein targets, ordinary stochastic sampling from the selected TTRL policy achieved higher mean normalized MFE than best-of-800 sampling from the starting RFT policy. Together with NanoLuc, these results support the claim that TTRL can further adapt a reusable RFT policy toward higher reward across multiple protein targets, beyond the results obtained by the evaluated sampling strategy from the frozen policy.

LinearDesign achieved higher normalized MFE, consistent with its explicit optimization of folding stability. However, our NanoLuc experiments show that improvement in this metric does not necessarily translate into greater protein output. The distinction is therefore broader than reward maximization alone: our framework uses a protein-conditioned model trained on natural human CDSs as a learned sequence prior, providing a basis for balancing desired sequence properties with proximity to the learned distribution. LinearDesign instead optimizes explicitly specified objectives without such a learned full-sequence prior, meaning it has no mechanism to trade reward against distributional plausibility.

Our NanoLuc wet-lab results provide experimental evidence of the framework’s utility; the additional targets extend the computational evidence for reward adaptation. Thus, based on wet-lab validation where both GEMORNA and especially Prot2RNA outperform LinearDesign despite lower MFE, the claim of underperformance of Prot2RNA on new targets based on MFE score only is incorrect. Experimental evaluation on these additional targets remains an important direction for future work.

Reviewer_GRQCOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents Prot2RNA, a protein-conditioned discrete diffusion framework for synonymous coding sequence (CDS) design. The model learns a conditional distribution over natural CDSs using masked diffusion modeling and a modality-aligned positional encoding scheme. The authors subsequently apply global reinforcement fine-tuning (RFT) using GRPO with normalized minimum free energy (MFE) as the reward, followed by test-time reinforcement learning (TTRL) for target-specific optimization. The approach demonstrates competitive performance against existing CDS generation methods in codon accuracy and distributional similarity. The reinforcement learning stages improve the optimized MFE reward, and preliminary NanoLuc experiments show higher reporter output for the selected TTRL design than for the tested LinearDesign and GEMORNA constructs.

Critical strengths
  1. The modality-aligned positional encoding explicitly exploits the one-to-one relationship between amino acids and codons. Although simple, it provides a substantial improvement in codon accuracy (0.688 versus 0.521 for standard RoPE; Table 2). This is a concrete task-specific design contribution that deserves recognition.

  2. The authors provide systematic comparisons of training objectives, reward optimization settings, and sampling strategies. The inclusion of a NanoLuc reporter assay is particularly valuable, as it provides preliminary experimental support beyond computational metrics. These results make the work practically relevant despite its limited algorithmic novelty.

Critical weaknesses
  1. The pipeline combines a RiNALMo-based encoder, LLaDA-style masked diffusion, MaskGIT-style iterative decoding, GRPO, and target-specific test-time RL. Each ingredient has a clear precedent, and related work already studies reward-guided discrete diffusion and RL-based codon design (Appendix A). The aligned positional encoding is useful and empirically effective, but its core idea follows directly from the known amino-acid/codon correspondence. The authors should more clearly explain which transferable learning insight, beyond effective task-specific integration, is new. Limited novelty should not be confused with an absence of meaningful engineering contribution.

  2. RFT and TTRL successfully shift the model toward higher normalized MFE, but the paper does not establish their optimization advantage over simpler synonymous-codon search. The Best-of-N experiments in Appendix N use sampling budgets that do not amount to a full, cost-matched comparison with local search, evolutionary strategies, or simulated annealing. The one-time global RFT cost should be reported separately and amortized over multiple targets, whereas the extra NanoLuc-specific TTRL search should be counted for that target. Moreover, the matched-distribution comparison in Figure 10 shows a pronounced FTD increase after RFT, alongside reduced diversity (Figure 3B), raising questions about how much reward gain is explained by simple compositional shifts. The single-target reporter assay is promising but insufficient to establish general superiority.

Questions for authors
  1. Which component, beyond the established LLaDA-style diffusion, MaskGIT decoding, GRPO, and TTRL, provides a generalizable methodological insight? In particular, how does the effective aligned RoPE design relate to the claimed broader contributions of the framework?

  2. Can the authors compare RFT/TTRL with local search, genetic algorithms, or simulated annealing at matched reward-evaluation budgets? Please distinguish the one-off global RFT cost (and its amortized cost across targets) from the additional target-specific TTRL cost.

  3. Can the authors quantify how much of the MFE improvement is explained by GC content or codon usage, using appropriate composition-matched controls? Please also clarify the reward–diversity–FTD trade-off in the matched-population analysis of Figure 10.

2 comments
AuthorsOpenReview ↗

Reply to Reviewer GRQC

We would like to thank Reviewer GRQC for taking the time to read our paper and for their encouraging comments and questions. We appreciate their recognition of a task-specific design contribution and their credit to the wet-lab experiments included in the results, which provide preliminary experimental support beyond computational metrics.

We agree that a single-target assay cannot establish general superiority. Given the scope of the experiment, we chose NanoLuc because it offers a sensitive readout of protein expression and a strong commercial reference that is already 100- to 150-fold brighter than traditional luciferase. Our method yielded higher reporter expression than the industry-standard Promega NanoLuc sequence and the sequences designed by the state-of-the-art methods we evaluated. This shows that our method can improve on established baselines for NanoLuc under the tested conditions. We will clarify the scope of this result in the manuscript.

Answering the Reviewer GRQC questions:

Question 1

Our main methodological insight is that explicitly encoding known cross-modal correspondences can substantially improve conditional generation, rather than relying on Transformers to learn these relationships implicitly. Aligning amino-acid and codon positions improves codon accuracy from 0.521 to 0.688. This inductive bias could generalize to other aligned biological modalities, such as RNA sequence and nucleotide-level secondary structure.

Moreover, we showed that a diffusion LM with aligned amino-acid and codon positions is better suited for CDS design when compared under matched architecture/data/compute against a next-token-prediction approach that is currently dominant in related work. This is a fondation that can translate to other aligned biological modalities that can benefit from bi-directional context.

Question 2

Motivation for global RFT and TTRL. Our goal is not solely to develop a more efficient optimizer for a particular reward, but to establish a framework in which a protein-conditioned diffusion model trained on natural human CDSs can be steered toward desired sequence-level properties. Global RFT incorporates reward optimization into a reusable generative policy, allowing reward-guided generation for previously unseen proteins without repeating optimization for each target. TTRL provides an additional mechanism for specializing this policy when a particular protein becomes the target of interest, especially when it differs from the proteins encountered during training. In our experiments, we excluded NanoLuc and similar sequences from SFT and RFT training, as ensured by our protein sequence clustering and cluster-level data splitting (Appendix F).

Additional local-search comparisons. To address the reviewer's question, we implemented greedy hill climbing (HC) and simulated annealing (SA) using the same normalized-MFE reward as RFT/TTRL. Both methods iteratively propose random synonymous codon substitutions while preserving the encoded protein. HC accepts only reward-improving substitutions, whereas SA also allows non-improving substitutions according to a geometrically decreasing temperature schedule.

Evaluation budgets and initialization. We first initialized both methods from an RFT-generated NanoLuc CDS, matching the initialization used for TTRL. We allocated 800 reward evaluations per run, corresponding to the evaluation budget of an individual TTRL configuration. Each method was repeated with 51 random seeds, matching the number of hyperparameter configurations explored during TTRL.

We additionally evaluated HC and SA starting from an SFT-generated NanoLuc CDS to examine optimization without global reward adaptation. Since global RFT learns a reusable policy across proteins rather than optimizing NanoLuc directly, we used the average number of reward evaluations per training protein as a reference budget. During RFT, each of the approximately 500 training proteins was sampled 32 times (batch size was 16), with 16 generated CDSs per occurrence, corresponding to 512 reward evaluations per protein. Adding the 800 evaluations allocated for target-specific optimization gives 1,312 evaluations per run for the SFT-initialized searches. This provides a per-protein reference for the global training effort.

Results. Starting from RFT, HC achieved a mean best normalized MFE of 0.558 across seeds, with a maximum of 0.592, while SA achieved 0.533 on average, with a maximum of 0.564. Starting from SFT, the corresponding means were 0.489 and 0.528 for HC and SA, respectively . These results demonstrate that direct synonymous-codon search is highly effective for maximizing normalized MFE, with the best local-search sequences exceeding the reward of our TTRL-selected NanoLuc design (0.498). They also support using the reward-adapted RFT policy as a useful initialization for target-specific sequence optimization.

To further characterize the resulting sequences, we evaluated their compatibility with the pretrained Prot2RNA distribution using masked-codon reconstruction NLL (negative log-likelihood; lower is better). The SFT-generated NanoLuc CDS had an NLL of 0.590, providing a reference for the learned prior. Global RFT increased this to 0.715, while the experimentally selected TTRL sequence retained a similar NLL of 0.703. In contrast, HC and SA increased the mean NLL to 1.115 and 1.318 when initialized from SFT, and 1.060 and 1.322 when initialized from RFT, respectively. Although local search achieved higher normalized MFE, these results suggest that it moves sequences further from the learned CDS distribution, whereas the selected TTRL design retains greater compatibility with the pretrained prior.

Interpretation and biological relevance. HC and SA directly optimize individual CDSs, whereas RFT learns a reusable reward-adapted generative policy, and TTRL further adapts this policy to a specific target while remaining compatible with the pretrained CDS prior. We do not claim that TTRL is more efficient than conventional local search at maximizing normalized MFE.

Importantly, normalized MFE is a biologically motivated reward, not a complete objective for mRNA design. LinearDesign, which uses lattice parsing and dynamic programming to optimize MFE and codon adaptation index (CAI), illustrates this limitation. In our NanoLuc experiments, LinearDesign achieves higher normalized MFE than Prot2RNA-TTRL (0.591 versus 0.498), yet produces lower reporter output. Similarly, GEMORNA reported higher experimental protein expression than LinearDesign for firefly luciferase and COVID-19 spike, despite substantially lower normalized MFE values (0.419 versus 0.670 and 0.370 versus 0.601, respectively). These results illustrate why optimizing MFE, even alongside CAI, does not guarantee higher protein output. Our contribution is therefore not simply maximizing MFE, but developing a reusable generative framework that learns from natural CDSs and can be adapted toward desired properties, with the potential to incorporate more biologically informative rewards as they become available.

Question 3

We agree that increased GC content contributes to the normalized MFE improvement. To investigate whether GC content or codon usage alone can explain these gains, we implemented two simple baselines: GC-max, which selects the most GC-rich synonymous codon at each position with random tie-breaking (three seeds), and CAI-max, which selects the highest-usage synonymous codon. Across held-out test proteins, these achieve median normalized MFE values of 0.450 and 0.419, respectively, compared with 0.477–0.479 for Prot2RNA-RFT.

For NanoLuc, we additionally generated 20 GC-max sequences using different random tie-breaking. Their normalized MFE ranged from 0.402 to 0.447, whereas Prot2RNA-TTRL achieved 0.496.

To further control for GC content, we selected 40 held-out proteins and, for each RFT-generated CDS, generated 100 random synonymous sequences encoding the same protein and matching the exact GC nucleotide count of that RFT-generated CDS. For β = 0, RFT achieved a mean normalized MFE of 0.477, compared with 0.449 for the GC-matched controls; for β = 0.005, the corresponding values were 0.474 and 0.448. RFT outperformed the mean of its GC-matched controls for 40/40 and 37/40 proteins, respectively. These results indicate that RFT's reward gains cannot be explained by total GC content alone, and that synonymous codon arrangement also contributes to predicted folding stability. This is expected because RNA secondary-structure stability depends not only on GC content but also on the arrangement of complementary nucleotides that can form favorable base-pairing interactions.

Regarding the shifts in FTD and diversity, Figure 10 shows that changing the regularisation parameter /beta which regularizes KL divergence between the base Prot2RNA model and optimized policies, changes the amount we can move from the learned conditional distribution. Giving less weight to the KL term, the model moves further from the wild-type CDS distribution, which is non-optimized for free energy. The only trade-off is between reward and diversity, which can be seen in Figure 3. Prot2RNA-RFT consistently shifts normalized MFE in the intended direction across held-out test proteins: median normalized MFE increases from 0.355 for Prot2RNA to 0.465−0.479 across the evaluated KL coefficients /beta (Figure3A). Weaker KL regularization permits stronger reward optimization but reduces within-protein sequence diversity (Figure 3B).

Reviewer_DFTwOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
3
Soundness
3
Read review
Paper summary

Authors train a discrete diffusion model to generate codon sequences underlying any protein. Codon sequences that translate into the same protein still differ in stability. The authors train their model in stages, starting with separately pre-training the model on protein and codon sequences in a masked language model fashion and then as pairs by fixing the protein and masking only the associated codon sequence, tailoring the architecture to function as a conditional masked diffusion language model. The third stage involves GRPO fine-tuning to optimize estimated free-energy scores of predicted RNAs across proteins, and potentially also further optimizing the codons for any new protein of interest. The technical approach is straightforward, borrowing existing ideas and methods from the literature. Comparisons do not include the latest competing methods apparently due to a lack of code availability. The presented comparisons show clear improvements (e.g., codon recovery). A wetlab demonstration suggests that the authors' approach leads to greater output even if another method (linearDesign) achieves better calculated free energy scores for this protein.

Critical strengths

Clearly written, straightforward, cleanly executed approach. The applied problem is interesting and influential. Included wet lab result for a single test protein is definitely a plus.

One technical modification is a natural change in positional encoding so that residues and their corresponding codons are "matched" and receive the same positional encoding.

Critical weaknesses

Little to no novelty in the technical approach overall, either in the discrete diffusion generative part or in the follow-on RL fine-tuning. Hard to assess significance in comparison to many recent approaches that are not included.

Questions for authors

How dependent are estimated free energy scores on the combinatorial choice of codons associated with different residue positions? It would be helpful to provide some indication of how challenging the generative task is.

3 comments
AuthorsOpenReview ↗

Reply to Reviewer DFTw

We would like to thank Reviewer DFTw for taking the time to read our paper and for their comments and questions. We appreciate their recognition of our writing style and clarity, and especially their appreciation of the wet-lab experiments included in the results, which are expensive and time-consuming but the only proper way to test the effectiveness of the mRNA design.

Response to weaknesses comments:

We position this as an application paper, with strong empirical contribution, and we accept that its components are individually established. We claim that reward-guided CDS design is currently a subfield in which proxy improvements are reported largely without the controls needed to attribute them, and that this paper supplies both those controls and a biological readout. We showed matched architecture/data/compute comparisons across training objectives, best-of-N controls at both adaptation scales, and introduced modality-aligned positional encoding that explicitly exploits the one-to-one relationship between amino acids and codons and a Fréchet-style population metric, FTD, to objectively measure semantic naturalness in the generated sequences. Additionally, we reported a cell-based panel against two independently validated published designs. Our Prot2RNA base model can serve as a strong learned prior for CDS design, coupled with different rewards to steer toward the desired sequence properties, and the Prot2RNA-RFT model can be used off-the-shelf to generate more stable sequences. We also provide data preparation, rigorous splits and evaluation, model weights, and training code. We believe this establishes a strong foundation and platform for future research directions in mRNA design.

In the paper, we compared our approach to the peer-reviewed, state-of-the-art methods for mRNA CDS design, LinearDesign [1] and GEMORNA [2]. We additionally compared the Prot2RNA base model (generative diffusion language model) to other generative CDS language models, such as peer-reviewed CodonBERT, CodonTransformer, and autoregressive Trias. In both cases, we showed our methods outperformed others, either on in silico metrics such as codon accuracy and Fréchet transcript distance or in wet-lab experiments measuring reporter output. For completeness, we mentioned other recent reinforcement learning-based methods developed in parallel with ours; most are not peer-reviewed, report only in silico metrics, and do not have open-source code.

In the revised version, we decided to compare our approach to two other peer-reviewed, RL-based methods, codonGPT [3] and RNARL [4]. Due to time constraints, we can compare these RL-based methods only in silico. If the reviewer has a specific method in mind that would help address this concern, we would appreciate a pointer to the relevant work.

As of 15 Nov 2026, several related methods lack publicly available implementations or are unsuitable as CDS design baselines. For codonGPT [3], only inference code and the Ensembl-pretrained base model are available, without training or RL/PPO code. For mRNA-GPT [5], neither code nor weights are available. For ProMORNA [6], we found no public implementation, weights, or inference code, and the preprint lacks a code-availability statement. mRNAutilus [7] uses inference-time Monte Carlo tree guidance over a frozen masked-diffusion model, requiring a template CDS and fixed UTR lengths. Its released implementation redesigns only 15% of CDS positions, leaving the rest unchanged, while also generating UTRs de novo. We therefore do not include it as a CDS design baseline.

CodonGPT results. Although the RL-trained CodonGPT model and its PPO implementation are not publicly available, we evaluated the released pretrained CodonGPT checkpoint on our held-out test proteins. Compared with CodonGPT, pretrained Prot2RNA achieves substantially higher mean codon accuracy (0.647 vs. 0.413) and lower Fréchet transcript distance (3.69 vs. 7.48), indicating better protein-conditioned codon reconstruction and closer agreement with the natural human CDS distribution in the evaluated feature space. Prot2RNA also achieves slightly higher median normalized MFE (0.368 vs. 0.344) before reinforcement finetuning. These comparisons concern the pretrained generative models; we do not claim an RL optimization advantage over CodonGPT's unreleased PPO-trained model.

RNARL results. Using the authors' publicly available Google Colab interface, we generated an optimized NanoLuc CDS and evaluated it using the same ViennaRNA-based normalized MFE metric as our designs. RNARL achieved 0.4409, compared with 0.4484–0.4558 for Prot2RNA-RFT and 0.4977 for Prot2RNA-TTRL. These results provide an additional in silico comparison on NanoLuc, although the methods optimize different reward objectives and this single-target evaluation does not establish overall superiority.

Answering the Reviewer DFTw questions:

First, we would like to emphasize the combinatorial space we need to find an optimal design from. The efficiency of mRNA translation depends heavily on the design of its CDS, where the choice of synonymous codons, referring to different codons that encode the same amino acid, can significantly impact protein output. Due to redundancy of the genetic code, where codons encode amino acids, a single protein can be encoded by an enormous number of alternative mRNA sequences that differ in codon composition but produce the same amino acid chain. For example, the 1,273-amino-acid SARS-CoV-2 spike protein can be coded in more than 1063210^{632} different synonymous ways [8]. However, only a small fraction of the sequences lead to high expression in human cells, a stable secondary structure, and favorable translational kinetics [9], [10]. Identifying such high-performing sequences remains a major computational challenge. In this manuscript we used the 171-amino-acid NanoLuc luciferase as a test case because it is a compact, widely used reporter whose strong luminescent signal allows us to quantify protein expression across synonymous coding variants. Even for a protein this short, roughly 108410^{84} distinct coding sequences can encode the same amino acid sequence.

To further investigate the difficulty of optimizing MFE through synonymous codon choices, we implemented a simple GC-max baseline, which independently selects the synonymous codon with the highest GC content at each amino acid position. When multiple codons have the same maximal GC content, we break ties uniformly at random. We generated sequences using three random seeds (42, 43, 44) for each held-out test protein.

Across the test set, GC-max achieves a median normalized MFE of 0.450, compared with 0.465-0.479 for Prot2RNA-RFT. We additionally examined the NanoLuc reporter, generating 20 GC-max sequences with different random tie-breaking. Despite having identical GC content, their normalized MFE ranged from 0.402 to 0.447, while our TTRL design reached 0.496. These results illustrate that RNA folding stability depends not only on GC content but also on the arrangement of complementary nucleotides and their potential base-pairing interactions. Thus, simply inflating GC content at each codon position is insufficient to identify the highest-reward designs.

References:

[1] Zhang, He, et al. "Algorithm for optimized mRNA design improves stability and immunogenicity." Nature 621.7978 (2023): 396-403.

[2] Zhang, He, et al. "Deep generative models design mRNA sequences with enhanced translational capacity and stability." Science 390.6773 (2025): eadr8470.

[3] Rajbanshi, Binita, and Anuj Guruacharya. "codonGPT: reinforcement learning on a generative language model enables scalable mRNA design." Nucleic Acids Research 53.22 (2025): gkaf1345.

[4] Lin, Shenggeng, et al. "RNARL: reinforcement learning-driven unified generative framework for multi-objective RNA codon design." Genome Biology (2026).

[5] Li, Sizhen, et al. "mRNA-GPT: A Generative Model for Full-Length mRNA Design and Optimization." bioRxiv (2026): 2026-03.

[6] Shao, Zixi, et al. "Protein-Conditioned Multi-Objective Reinforcement Learning for Full-Length mRNA Design." arXiv preprint arXiv:2605.01513 (2026).

[7] Patel, Sawan, et al. "mRNAutilus: Multi-Objective-Guided Discrete Generation of mRNA with Optimized Therapeutic Properties." arXiv preprint arXiv:2605.31296 (2026).

[8] Kim, Yoo-Ah, et al. "Computational design of mRNA vaccines." Vaccine 42.7 (2024): 1831-1840.

[9] Walsh, Ian M., et al. "Synonymous codon substitutions perturb cotranslational protein folding in vivo and impair cell fitness." Proceedings of the National Academy of Sciences 117.7 (2020): 3528-3534.

[10] Moss, McKenze J., Laura M. Chamness, and Patricia L. Clark. "The effects of codon usage on protein structure and folding." Annual Review of Biophysics 53 (2024).

I appreciate the many clarifications, including baselines (e.g., simple GC-max that achieves comparatively high MFE). I maintain my original assessment.

Comments on the paper

4 comments
AuthorsOpenReview ↗

Major changes and improvements in the paper and rebuttal summary

We would like to thank the reviewers for their comments and constructive feedback. We considered the comments, made several additional in silico experiments and revised the paper. We believe the paper is improved and now more suited to be published.

Overview:

We position this as an application-driven ML contribution to mRNA design, supported by strong empirical evidence: we accept that its components are individually established, and our contribution is to develop and validate a framework for optimizing protein-conditioned CDSs while leveraging a learned prior over natural human sequences. Reward-guided CDS design is a growing field in which improvements in computational proxies are often reported without sufficient controls to establish their relationship to biological outcomes. This paper supplies both rigorous computational controls and a biological readout: matched architecture/data/compute comparisons across training objectives, best-of-N controls at both adaptation scales, and a cell-based panel against two independently validated published designs. The paper further proposes two contributions of broader use: a modality-aligned positional encoding that explicitly exploits the one-to-one amino-acid/codon correspondence (codon accuracy 0.521 → 0.688, an inductive bias that could transfer to other aligned biological modalities), and FTD, a Fréchet-style population metric for similarity to natural coding sequences. Crucially, the application-level gains are not confined to computational proxies: in HEK293T cells, the selected NanoLuc design achieves the highest measured reporter output among the tested constructs, 39% and 71% higher than GEMORNA and LinearDesign, respectively. We release data preparation, splits, evaluation, model weights, and training code; the Prot2RNA base model can serve as a reusable protein-conditioned prior for optimization toward different sequence-level objectives, and Prot2RNA-RFT provides an off-the-shelf policy optimized for greater predicted structural stability. We believe this establishes a practical and experimentally supported foundation for ML-driven mRNA design, with broader potential for adapting learned biological sequence models to application-specific objectives.

List all the improvements and experiments we ran.

  1. Normalized free energy vs. naive GC content increase, as a reply to Reviewer DFTw and Reviewer GRQC

We agree that increased GC content contributes to the normalized MFE improvement. To investigate whether GC content or codon usage alone can explain these gains, we implemented two simple baselines: GC-max, which selects the most GC-rich synonymous codon at each position with random tie-breaking (three seeds), and CAI-max, which selects the highest-usage synonymous codon. Across held-out test proteins, these achieve median normalized MFE values of 0.450 and 0.419, respectively, compared with 0.477–0.479 for Prot2RNA-RFT.

We additionally examined the NanoLuc reporter, generating 20 GC-max sequences with different random tie-breaking. Despite having identical GC content, their normalized MFE ranged from 0.402 to 0.447, while our Prot2RNA-TTRL design reached 0.496.

To further control for GC content, we selected 40 held-out proteins and, for each RFT-generated CDS, generated 100 random synonymous sequences encoding the same protein and matching the exact GC nucleotide count of that RFT-generated CDS. For β = 0, RFT achieved a mean normalized MFE of 0.477, compared with 0.449 for the GC-matched controls; for β = 0.005, the corresponding values were 0.474 and 0.448. RFT outperformed the mean of its GC-matched controls for 40/40 and 37/40 proteins, respectively.

These results indicate that RFT's reward gains cannot be explained by total GC content alone, and that synonymous codon arrangement also contributes to predicted folding stability. This is expected because RNA secondary-structure stability depends not only on GC content but also on the arrangement of complementary nucleotides that can form favorable base-pairing interactions.

In the revised paper, we added Figures 11 and 12 and discussed the simple baseline on page 21 (Appendix N).

  1. In silico comparison to other methods, as a reply to Reviewer DFTw

We compared our approach to the peer-reviewed, state-of-the-art methods for mRNA CDS design, LinearDesign and GEMORNA. We also compared our Prot2RNA base model with other generative CDS language models, including the peer-reviewed CodonBERT, CodonTransformer, and autoregressive Trias. Our methods outperformed these baselines on in silico metrics or experimental reporter output. For completeness, we mentioned other recent reinforcement learning-based methods developed in parallel with ours; most are not peer-reviewed, report only in silico metrics, and do not have open-source code.

In the revised version, we compared our approach with two other peer-reviewed, RL-based methods: codonGPT [1], RNARL [2] and CodonRL [3]. Due to time constraints, we can compare these RL-based methods only in silico. Note that for proper evaluation, we would need to do wet-lab experiments, which remains an important direction for future work.

CodonGPT results. Although the RL-trained CodonGPT model and its PPO implementation are not publicly available, we evaluated the released pretrained CodonGPT checkpoint on our held-out test proteins (n=3,004). Compared with CodonGPT, pretrained Prot2RNA achieves substantially higher mean codon accuracy (0.647 vs. 0.413) and lower Fréchet transcript distance (3.69 vs. 7.48), indicating better protein-conditioned codon reconstruction and closer agreement with the natural human CDS distribution in the evaluated feature space. The Prot2RNA base model also achieves a slightly higher median normalized MFE (0.368 vs. 0.344). These comparisons concern the pretrained generative models and we do not claim an RL optimization advantage over CodonGPT's unreleased PPO-trained model.

RNARL results. Using the authors' publicly available Google Colab interface, we generated optimized NanoLuc, EPO and eGFP coding sequences and evaluated them using the same ViennaRNA-based normalized MFE metric as our designs. The table compares our model with RNARL. These results provide an additional in silico comparison on held-out target proteins, although the methods optimize different reward objectives and this single-target evaluation does not establish overall superiority.

CodonRL results. CodonRL trains a separate Q-network per target protein, with a LinearDesign (λ = 0) guide trajectory, which makes its configuration a target-specific optimizer, suitable for comparison with our Prot2RNA-TTRL. Its public implementation additionally provides a uniform weight average of the 55 released per-protein checkpoints, a model soup [4], yielding a single protein-conditioned policy that can be applied to unseen proteins without further optimization. This configuration is the regime-matched counterpart to Prot2RNA-RFT, and we evaluate it as such. First, we evaluated the CodonRL model soup on our held-out test proteins. When compared, Prot2RNA-RFT achieves substantially higher mean normalized MFE values (0.4796 vs. 0.3769), meaning the CodonRL model soup cannot achieve high normalized MFEs for proteins its single-target policy was not trained on. We additionally generated optimized NanoLuc, EPO and eGFP coding sequences using the CodonRL model soup and default target-specific CodonRL and evaluated them. The results are provided in the table below.

Reward-only CodonRL optimizes a design objective without a model of the natural CDS distribution, so it has no mechanism to trade reward against distributional plausibility. This and LinearDesign (λ = 0) guide trajectory lead CodonRL design to the highest normalized MFE values. However, in our panel, the construct with the highest normalized MFE did not produce the highest reporter output, consistent with normalized MFE capturing only one component of CDS design. Finally, sweeping LinearDesign’s exact dynamic program over the same λ grid traces a frontier that no CodonRL configuration reaches under the same objective function, on any of the three targets: for every trained configuration there exists a λ for which LinearDesign’s solution is at least as good in both MFE and CAI. LinearDesign is exactly optimizable, and reinforcement learning recovers only an interior region of the same trade-off surface at substantially greater cost. In our case, there is no such solver, since we are optimizing a sequence-level reward while remaining close to a learned distribution over natural coding sequences.

Normalized MFE (higher is better)NanoLuceGFPEPO
RFT best-of-8000.47220.45050.5509
TTRL policy mean0.48370.46010.5996
TTRL best during search0.49770.46330.6056
RNARL0.44090.39930.5240
CodonRL model soup0.38550.34620.4171
CodonRL0.66780.64720.6985

In the revised paper, we added these results in Figures 2 and 3, and Tables 3 and 5, and discussed the baselines on page 8.

  1. Greedy hill climbing (HC) and simulated annealing (SA) vs. Prot2RNA-TTRL, as a reply to Reviewer GRQC

Our goal is not solely to develop a more efficient optimizer for a particular reward, but to establish a framework in which a protein-conditioned model trained on natural human CDSs can be steered toward desired sequence-level properties while remaining close to a learned distribution. Global RFT incorporates reward optimization into a reusable generative policy, allowing reward-guided generation for previously unseen proteins without repeating optimization for each target. TTRL provides an additional mechanism for specializing this policy when a particular protein becomes the target of interest, especially when it differs from the proteins encountered during training.

To address the reviewer's question, we implemented greedy hill climbing (HC) and simulated annealing (SA) using the same normalized-MFE reward as RFT/TTRL. Both methods iteratively propose random synonymous codon substitutions while preserving the encoded protein. HC accepts only reward-improving substitutions, whereas SA also allows non-improving substitutions according to a geometrically decreasing temperature schedule.

Evaluation budgets and initialization. Both methods were first initialized from an RFT-generated NanoLuc CDS, matching the initialization and budget used for TTRL. We additionally ran HC and SA from an SFT-generated CDS to examine optimization without global reward adaptation. Since global RFT learns a reusable policy across proteins rather than optimizing NanoLuc directly, we used its amortized per-protein cost as a reference.

Results. Starting from RFT, HC achieved a mean best normalized MFE of 0.558 across seeds, with a maximum of 0.592, while SA achieved 0.533 on average, with a maximum of 0.564. Starting from SFT, the corresponding means were 0.489 and 0.528 for HC and SA, respectively. These results demonstrate that direct synonymous-codon search is highly effective for maximizing normalized MFE, with the best local-search sequences exceeding the reward of our TTRL-selected NanoLuc design (0.498). They also support using the reward-adapted RFT policy as a useful initialization for target-specific sequence optimization.

To further characterize the resulting sequences, we evaluated their compatibility with the pretrained Prot2RNA distribution using masked-codon reconstruction NLL (negative log-likelihood; lower is better). The SFT-generated NanoLuc CDS had an NLL of 0.590, providing a reference for the learned prior. Global RFT increased this to 0.715, while the experimentally selected TTRL sequence retained a similar NLL of 0.703. In contrast, HC and SA increased the mean NLL to 1.115 and 1.318 when initialized from SFT, and 1.060 and 1.322 when initialized from RFT, respectively. Although local search achieved higher normalized MFE, these results suggest that it moves sequences further from the learned CDS distribution, whereas the selected TTRL design retains the desired greater compatibility with the pretrained prior.

Interpretation and biological relevance. HC and SA directly optimize individual CDSs, whereas RFT learns a reusable reward-adapted generative policy, and TTRL further adapts this policy to a specific target while remaining compatible with the pretrained CDS prior.

Importantly, normalized MFE is a biologically motivated reward, not a complete objective for mRNA design. LinearDesign, which uses lattice parsing and dynamic programming to optimize MFE and codon adaptation index (CAI), illustrates this limitation. In our NanoLuc experiments, LinearDesign achieves higher normalized MFE than Prot2RNA-TTRL (0.591 versus 0.498), yet produces lower reporter output. Similarly, GEMORNA reported higher experimental protein expression than LinearDesign for firefly luciferase and COVID-19 spike, despite substantially lower normalized MFE values (0.419 versus 0.670 and 0.370 versus 0.601, respectively). These results illustrate why optimizing MFE, even alongside CAI, does not guarantee higher protein output. Our contribution is therefore not simply maximizing MFE, but developing a reusable generative framework that learns from natural CDSs and can be adapted toward desired properties, with the potential to incorporate more biologically informative rewards as they become available.

In the revised paper, we mention these results in Section 3.3 (p. 8) and discuss them in the Discussion (p. 10), with the full comparison and methodology provided in Appendix Q (pp. 23–24, Figure 14).

  1. Best-of-N Prot2RNA-RFT vs. Prot2RNA-TTRL under the same budget, as a reply to Reviewer L21m

We extended best-of-N sampling from the frozen RFT policy to N = 49,200, matching the full NanoLuc TTRL search budget. The expected best normalized MFE increases from 0.4722 at N = 800 to only 0.4785 at N = 49,200, compared with 0.4977 for the TTRL-selected sequence and a mean of 0.484 across 64 samples from the selected TTRL policy.

Thus, even with the full target-specific evaluation budget, additional sampling from the frozen RFT policy does not reach the reward achieved through TTRL. This shows that target-specific policy adaptation can access higher-reward designs than increasing the original RFT policy's sampling budget.

In the revised paper, we report the full-budget comparison in Appendix O.2 (p. 22).

  1. More targets in silico with Prot2RNA-TTRL, as a reply to Reviewer L21m

We extended our in silico TTRL evaluation to two additional targets: an eGFP-based reporter (284 aa) from the Moderna study [5] and an engineered erythropoietin (EPO) construct (221 aa) from the GEMORNA paper. Neither target nor closely related protein sequences were represented in our SFT or global RFT training sets.

In silico results for the additional two targets are as follows:

Normalized MFE (higher is better)eGFPEPO
RFT best-of-8000.45050.5509
TTRL policy mean0.46010.5996
TTRL best during search0.46330.6056
GEMORNA0.33290.4733
LinearDesign (λ = 3.3)0.57610.6817

For context, the highest normalized MFE among 30 published Moderna eGFP constructs was 0.4095, while the highest among five published GEMORNA EPO designs was 0.4913. Prot2RNA-TTRL exceeded the RNARL and GEMORNA-generated designs on this objective, whereas LinearDesign achieved higher normalized MFE, consistent with its explicit optimization of RNA folding stability. As discussed in our NanoLuc experiments, however, higher normalized MFE does not necessarily translate into greater protein output. Together with NanoLuc, these results demonstrate that TTRL can further shift a reusable RFT policy toward higher reward across multiple protein targets, beyond what is achieved through additional sampling from the frozen policy. Experimental evaluation on these additional targets remains an important direction for future work.

In the revised paper, we report these results in Table 5 (p. 8), discuss them in Section 3.3 (p. 7-8), and provide additional best-of-N analyses in Appendix P (p. 23, Figure 13).

References:

[1] Rajbanshi, Binita, and Anuj Guruacharya. "codonGPT: reinforcement learning on a generative language model enables scalable mRNA design." Nucleic Acids Research 53.22 (2025): gkaf1345.

[2] Lin, Shenggeng, et al. "RNARL: reinforcement learning-driven unified generative framework for multi-objective RNA codon design." Genome Biology (2026).

[3] Du, Shiyi, et al. "CodonRL: Multi-Objective Codon Sequence Optimization Using Demonstration-Guided Reinforcement Learning." bioRxiv (2026).

[4] Wortsman, Mitchell, et al. "Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference time." ICML. PMLR, 2022.

[5] Bicknell, Alicia A., et al. "Attenuating ribosome load improves protein output from mRNA by limiting translation-dependent mRNA decay." Cell Reports 43.4 (2024).

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.