Exploratory Biological Sequence Design on Masked Discrete Diffusion Models

Biological sequence design under limited experimental budgets requires effective exploration of vast discrete sequence spaces under unreliable proxies trained from scarce data. Conservative search may remain concentrated around previously observed sequences, whereas broader exploration may reach regions where proxy predictions are uncertain. To reconcile conservative and broad exploration, we introduce a masked discrete diffusion framework Reward-tilted Diverse Discrete Diffusion (RD3). Our method consists of three components to precisely control exploration. First, it constructs local neighborhoods of high-scoring sequences by partial masking and denoising. Second, we fine-tune the pretrained masked diffusion model to tilt sequence towards high-scoring regions during denoising. Finally, we apply categorical particle guidance, a variant of particle guidance in continuous space to promote diversity in the batch. Experimental results across nine benchmarks spanning DNA, RNA, peptide, and protein design demonstrate that RD3 effectively discovers high-fitness sequences, outperforming existing methods in terms of maximum fitness. Fitness–diversity and fitness–novelty analyses further demonstrate the effectiveness of RD3 in generating diverse and novel candidates without compromising fitness. Overall, the results highlight the value of combining search locality, reward alignment, and finite-sample diversification to balance fitness, novelty, and diversity under limited oracle budgets.