- Rating
- 1(−1)
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper introduces Prot2RNA, a pipeline consisting of a discrete diffusion model trained for mRNA CDS generation, RL finetuning with GRPO to maximize minimum free energy (MFE), and test-time reinforcement learning with fixed protein targets. The method is validated in silico and in vitro in HEK293T cells, showing higher reporter output compared to published baselines.
Critical strengths
- The empirical results are solid with some wet-lab validation, showing improvements against published baselines like GEMORNA.
Critical weaknesses
- There is no clear theoretical or algorithmic novelty, and the algorithm seems to be a simple combination of discrete diffusion and RL finetuning with GRPO. It also doesn’t cite or compare against several relevant baselines, such as [1] or [2]. My current stance on this paper is that it is unsuitable for publication at an ML conference like ICLR.
- Validation was performed on NanoLuc only and no other targets.
[1] Zekri, O., & Boullé, N. (2025). Fine-tuning discrete diffusion models with policy gradient methods.
[2] Tang, S., Zhu, Y., Tao, M., & Chatterjee, P. (2025). Tr2-d2: Tree search guided trajectory-aware fine-tuning for discrete diffusion.
Questions for authors
- The normalized MFE and NanoLuc output seem weakly correlated in Figure 5, is there a way to distinguish the reason for the gain in NanoLuc output in Prot2RNA over the designs from LinearDesign, which has the highest normalized MFE?
- What is the total number of reward evaluations used to design the best TTRL sequence, and how does it compare to best-of-N at an equal budget?
- Can the authors show improved performance for more targets other than NanoLuc, at least in silico, with TTRL?
4 comments
Question 3
We extended our in silico TTRL evaluation to two additional targets: an eGFP-based reporter (284 aa) from the Moderna study [5] and an engineered erythropoietin (EPO) construct (221 aa) from GEMORNA [3]. Neither target, nor closely related protein sequences, was represented in the SFT or global RFT training sets.
Starting from Prot2RNA-RFT, we performed target-specific reinforcement learning using the same normalized-MFE reward as for NanoLuc. To distinguish improvements through policy adaptation from increased sampling of the starting RFT policy, we compared TTRL against best-of-N selection from the frozen RFT policy at a matched budget of 800 reward evaluations per run.
| Normalized MFE (higher is better) | eGFP | EPO |
|---|---|---|
| RFT best-of-800 | 0.4505 | 0.5509 |
| TTRL policy mean | 0.4601 | 0.5996 |
| TTRL best during search | 0.4633 | 0.6056 |
| RNARL [6] | 0.3993 | 0.5240 |
| GEMORNA | 0.3329 | 0.4733 |
| LinearDesign (λ = 3.3) | 0.5761 | 0.6817 |
For both targets, ordinary stochastic sampling from the selected TTRL policy achieved higher mean normalized MFE than best-of-800 sampling from the starting RFT policy. The highest-reward sequences identified during TTRL reached 0.4633 for eGFP and 0.6056 for EPO, demonstrating additional reward improvement through target-specific policy adaptation.
For context, the highest normalized MFE among 30 published Moderna eGFP constructs was 0.4095, while the highest among five published GEMORNA EPO designs was 0.4913. TTRL also exceeded the RNARL [6] and GEMORNA-generated designs on this objective, whereas LinearDesign achieved higher normalized MFE, consistent with its explicit optimization of RNA folding stability. As discussed in our NanoLuc experiments, however, higher normalized MFE does not necessarily translate into greater protein output.
Together with NanoLuc, these results demonstrate that TTRL can further shift a reusable RFT policy toward higher reward across multiple protein targets, beyond what is achieved through additional sampling from the frozen policy. Experimental evaluation on these additional targets remains an important direction for future work.
References:
[1] Zekri, O., and N. Boullé. "Fine-tuning discrete diffusion models with policy gradient methods." Advances in Neural Information Processing Systems 38 (2026): 152868–152906.
[2] Tang, S., et al. "TR2-D2: Tree search guided trajectory-aware fine-tuning for discrete diffusion." arXiv preprint (2025).
[3] Zhang, He, et al. "Deep generative models design mRNA sequences with enhanced translational capacity and stability." Science 390.6773 (2025): eadr8470.
[4] Zhang, He, et al. "Algorithm for optimized mRNA design improves stability and immunogenicity." Nature 621.7978 (2023): 396-403.
[5] Bicknell, Alicia A., et al. "Attenuating ribosome load improves protein output from mRNA by limiting translation-dependent mRNA decay." Cell Reports 43.4 (2024).
[6] Lin, Shenggeng, et al. "RNARL: reinforcement learning-driven unified generative framework for multi-objective RNA codon design." Genome Biology (2026).
Thank you for the rebuttal. However, my main concerns regarding theoretical novelty and limited evaluation remain. I argue that the suggested baselines can be easily adapted to protein-conditioned CDS generation by swapping out the rewards and finetuning the same base model. In addition, it seems like the added results on eGFP and EPO do not outperform LinearDesign. Given these limitations, I recommend reject.
Reply to Reviewer L21m
Dear Reviewer L21m,
We appreciate your suggestions and would like to clarify the contribution and the purpose of our comparisons. Our contribution is a framework for adapting a protein-conditioned generative model to mRNA design objectives; we do not propose a new reinforcement learning algorithm. GRPO serves as the optimization method for adapting the model to sequence-level rewards. Accordingly, GEMORNA, CodonBERT, Trias, and CodonTransformer provide application-level comparisons, while comparisons against the starting and frozen policies assess the benefit of adaptation. Evaluating GRPO against SEPO or TR2-D2 would address a different question—which policy optimization algorithm performs best within this framework—and we make no claim of superiority on that question.
We agree that discrete diffusion and GRPO are established methods. The contribution lies in how these components address mRNA sequence design and in the evidence supporting the resulting framework. We therefore ask that the work be assessed against its application objective and empirical findings. This is consistent with the ICLR Reviewer Guide, which explicitly recognizes better addressing a known application or problem as a valid research objective and states that different objectives require different considerations of value and impact. https://www.iclr.cc/Conferences/2027/ReviewerGuidelines
For both additional protein targets, ordinary stochastic sampling from the selected TTRL policy achieved higher mean normalized MFE than best-of-800 sampling from the starting RFT policy. Together with NanoLuc, these results support the claim that TTRL can further adapt a reusable RFT policy toward higher reward across multiple protein targets, beyond the results obtained by the evaluated sampling strategy from the frozen policy.
LinearDesign achieved higher normalized MFE, consistent with its explicit optimization of folding stability. However, our NanoLuc experiments show that improvement in this metric does not necessarily translate into greater protein output. The distinction is therefore broader than reward maximization alone: our framework uses a protein-conditioned model trained on natural human CDSs as a learned sequence prior, providing a basis for balancing desired sequence properties with proximity to the learned distribution. LinearDesign instead optimizes explicitly specified objectives without such a learned full-sequence prior, meaning it has no mechanism to trade reward against distributional plausibility.
Our NanoLuc wet-lab results provide experimental evidence of the framework’s utility; the additional targets extend the computational evidence for reward adaptation. Thus, based on wet-lab validation where both GEMORNA and especially Prot2RNA outperform LinearDesign despite lower MFE, the claim of underperformance of Prot2RNA on new targets based on MFE score only is incorrect. Experimental evaluation on these additional targets remains an important direction for future work.
Reply to Reviewer L21m
We thank Reviewer L21m for taking the time to review our manuscript and for their constructive feedback. We sincerely appreciate their recognition of our strong empirical improvements over existing baselines such as GEMORNA, as well as the value of our wet-lab validation in providing experimental support beyond computational metrics that are often lacking in the related work.
Response to weaknesses comments:
We thank the reviewer for highlighting SEPO [1] and TR2-D2 [2] and we are happy to include both in the related work. These methods propose alternative reinforcement fine-tuning algorithms for discrete diffusion, evaluated on tasks such as regulatory DNA and peptide design, rather than protein-conditioned CDS generation. A controlled quantitative comparison would require adapting their optimization algorithms to our protein-conditioned, synonymously constrained diffusion model. Our work does not claim algorithmic superiority over alternative RL methods, nor do we claim superiority of GRPO against other policy optimization methods; instead, it investigates the use of discrete diffusion for CDS generation and its adaptation to sequence-level rewards, supported by computational comparisons with established CDS-design methods and preliminary experimental validation. While we agree it would be interesting to compare RL methods such as SEPO, TR2-D2 and GRPO, we believe it is out of scope for this paper.
Answering the Reviewer L21m questions:
Question 1
Normalized MFE is not a direct predictor of protein output, as discussed in Sections 3.4 and 4. For example, in the GEMORNA paper [3], we can also see that GEMORNA outperformed LinearDesign [4] in experimental expression of firefly luciferase and COVID-19 spike, despite substantially lower normalized MFE values (0.419 vs. 0.670 and 0.370 vs. 0.601, respectively). Protein output depends on multiple interacting factors, including transcript stability, translation dynamics, and translation-dependent decay. Although the LinearDesign construct achieves the highest normalized MFE, this does not necessarily translate into higher reporter output. The hypothesis was that Prot2RNA preserves favorable sequence properties learned from natural CDSs while optimizing structural stability. In silico metrics are not directly transferable to the experimental total protein output. Wet-lab experiments, which are expensive and time-consuming, are the only proper way to test the effectiveness of the mRNA design.
Question 2
Our original best-of-N comparison used N = 800, matching the reward-evaluation budget of a single TTRL configuration (50 optimization steps, with 16 generated sequences per step). This comparison was intended to assess whether updating the generative policy provides an advantage over additional sampling from the frozen RFT policy.
For NanoLuc, we performed a staged hyperparameter search because suitable test-time adaptation settings were not known a priori. We selected hyperparameters based on the mean reward of generated sequences at the final training step, reflecting the adapted policy's performance rather than a single high-reward sample. We then selected the highest-reward CDS encountered during the chosen configuration's training trajectory for experimental evaluation.
We agree that the full hyperparameter-search budget should also be considered. Across the 51 configurations explored, the total target-specific cost was 49,200 reward evaluations. We therefore extended best-of-N sampling from the frozen RFT policy to N = 49,200, matching the full NanoLuc TTRL search budget. The expected best normalized MFE increases from 0.4733 at N = 800 to only 0.4785 at N = 49,200, compared with 0.4977 for the TTRL-selected sequence and a mean of 0.484 across 64 samples from the selected TTRL policy.
Thus, even with the full target-specific evaluation budget, additional sampling from the frozen RFT policy does not reach the reward achieved through TTRL. This demonstrates that target-specific policy adaptation can access higher-reward designs than those found by increasing the sampling budget of the original RFT policy.