- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 3
- Soundness
- 1
Read review
Paper summary
This paper introduces PILOT-UTR, which applies reward-finetuning to discrete diffusion for mRNA 5’UTR design, with calibrated rewards that claims to avoid reward hacking. The reward function is calibrated on OOD sequences generated with vanilla reward finetuning, and then used in finetuning with KL regularization. Experiments show improvements on independent rewards and metrics evaluating naturalism.
Critical strengths
- PILOT-UTR does show evidence of preserving naturalness, with the highest 3-mer correlation values.
- The paper aims to tackle reward hacking, which is an important problem in biological sequence design.
Critical weaknesses
- The calibrated reward model makes assumptions on the reward on OOD sequences, whereas, the typical goal is to explore OOD with a generative model and then calibrate the reward model with true experimental data in an active learning loop. This method seems to be unsuited for further downstream applications where experimental validation is considered.
- Scope on 5’UTR design is too narrow when the method can theoretically be applied to many modalities with hackable rewards, and relevant baselines/related works are not cited (e.g. [1], [2], [3], and [4]).
[1] Cao et al. (2025). GLIDE: A gradient-free lightweight fine-tune approach for discrete biological sequence design.
[2] Tang et al. (2025). Tr2-d2: Tree search guided trajectory-aware fine-tuning for discrete diffusion.
[3] Patel et al. (2026). mRNAutilus: Multi-Objective-Guided Discrete Generation of mRNA with Optimized Therapeutic Properties.
[4] De Santi et al. (2026). Active Flow Expansion for Out-of-Distribution Discovery: from Theory to Molecules.
Questions for authors
- How does this framework apply to other modalities (e.g. proteins, peptides, small molecules, natural language?). Focusing on mRNA 5’UTR sequence seems too narrow a scope for a top ML conference venue like ICLR.
- How does this compare to simply filtering out or performing negative finetuning on rollouts that result in OOD sequences during the finetuning process, rather than calibrating the reward?